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Overview of the NC Pre-K Program 


NC Pre-K is a state-funded educational program for eligible 4-year-olds, designed to enhance their 
school readiness skills. Initiated in 2001-2002, the program became statewide by 2003-2004." Since its 
inception, the statewide pre-k program has served over 400,000 children. According to program 
guidelines,’ children are eligible for NC Pre-K primarily based on age and family income. Children must 
be four years old by August 31 of the program year, with a gross family income at or below 75% of state 
median income (SMI). Within a local program, up to 20% of age-eligible children with higher family 
incomes may be enrolled if the child has at least one of the following additional factors: limited English 
proficiency, identified developmental disability, chronic health condition, or educational need (based on 
developmental screening or an IEP). In addition, children with a parent serving in the military are 
eligible regardless of family income or other eligibility factors. Programs also are encouraged to serve 
children in families who are experiencing homelessness. NC Pre-K provides funding for serving eligible 
children in classroom-based educational programs in a variety of setting types, including public schools, 
Head Start, and private child care centers (both for-profit and nonprofit). 


The requirements for NC Pre-K are designed to provide a high-quality, classroom-based educational 
experience for children, and to ensure uniformity in the program across the state, to the extent possible. 
The NC Pre-K Program operates on a school day and school calendar basis for 6-1/2 hours/day and 36 
weeks/year. Local sites are expected to meet a variety of program standards around curriculum, 
screening and assessment, training and education levels for teachers and administrators, class size, 
adult:child ratios, North Carolina child care licensing levels, and provision of other program services.' 
Class sizes are restricted to 18 children with a lead and assistant teacher, with adult:child ratios of 1:9. 
Lead teachers are required to hold or be working toward a NC Birth through Kindergarten (B-K) license 
or the equivalent and assistant teachers are required to hold or be working toward an Associate Degree 
in early childhood education or child development (ECE/CD) or a Child Development Associate (CDA) 
credential. Classroom activities and instruction are based on the state early learning standards" and an 
approved curriculum; classroom staff are expected to conduct developmental screenings and ongoing 
assessments to gather information on individual children’s growth and skill development as well as to 
inform instruction. Monthly reimbursement rates by the NC Pre-K Program vary by the type of 
classroom and teacher qualifications, from up to $400 per child (in Head Start sites) to a maximum of 
$650 (private sites with a B-K-licensed lead teacher), with an average annual cost per child estimated at 
$5,534, representing 61% of the total cost of $9,126. '! 


The program is administered at the state level by the Division of Child Development and Early Education 
(DCDEE), NC Department of Health and Human Services (NC DHHS), which then contracts with county or 
multi-county administrators to oversee local implementation. Contract administrators are primarily 
either local public school systems or local Smart Start partnerships, and must include collaboration 
among members of the local early childhood community (e.g., local public school systems, local Smart 
Start partnerships, Head Start, child care providers, resource and referral agencies) through an oversight 
committee. 


1|n 2011, the North Carolina General Assembly transferred the existing state pre-k program from the Department of Public Instruction (DPI) to the 
Division of Child Development and Early Education (DCDEE) in the North Carolina Department of Health and Human Services (DHHS) and renamed it from 
the More at Four Pre-Kindergarten Program to the North Carolina Pre-Kindergarten Program. 


Purpose of the NC Pre-K Evaluation — Randomized Controlled Trial Study 


The FPG Child Development Institute has conducted independent evaluation studies of the NC Pre-K 
Program (formerly More at Four) since its inception in the 2001-2002 school year. These evaluations 
have included multiple studies of program characteristics, classroom quality, and children’s outcomes 
over the course of the pre-k year as well as longitudinally into kindergarten and third grade. The 
evaluation designs have used a variety of methodologies, including pre and post, two-group comparison, 
regression discontinuity, and propensity score matching to examine both short-term and long-term 
outcomes. (See Appendix 1 for a list of previous reports.) 


The primary purpose of the 2017-2018 NC Pre-Kindergarten (NC Pre-K) Evaluation was to examine the 
effectiveness of the NC Pre-K Program using a randomized controlled trial (RCT) design. The major issue 
addressed by this type of study is around the benefits of offering children access to enrollment in NC 
Pre-K. An RCT is considered the gold-standard design for addressing such a question because children 
are randomly assigned to either receive the program (treatment) or not (control). Therefore, 
differences in outcomes can be causally attributed to whether or not they received treatment rather 
than to other differences between the children and families in the two different groups. 


This study is designed to follow children longitudinally from pre-k into elementary school in order to 
examine the short- and longer-term effects. The current study (2017-2018) provided baseline data 
about children’s outcomes during their pre-k year. This small-scale study compared 582 children who 
were randomly assigned to either NC Pre-K (Treatment=473) or the waitlist (Control=109) in two 
selected counties with substantially large waitlists. Because children in both groups had applied to and 
were eligible for NC Pre-K, they had similar characteristics. However, because not everyone could be 
served with the available number of program slots, random assignment was used to select children for 
NC Pre-K or the waitlist (with very few crossovers between groups). The study considered factors that 
might affect the impact of receiving the treatment — specifically, children’s level of oral language 
proficiency and classroom quality for children in preschool settings. In addition, the study also included 
a sample of children who were Spanish-speaking dual language learners and gathered data in both 
Spanish and English to examine the effects of treatment in both their first and second languages. 


The RCT study addressed three primary research questions: 


1) Dochildren who receive access to enrollment in NC Pre-K (treatment group) exhibit better 
outcomes than children who do not receive access to enrollment (control group)? 


2) Are there factors that affect the impact of treatment — children’s level of oral language 
proficiency or pre-k classroom quality? 


3) What are the effects for children who are dual-language learners? 


Data collection included individual assessments of children’s language, literacy, math, executive 
function, and behavior skills. Demographic information about the children and families was obtained 
from state administrative data and parent surveys, and observations of classroom quality and teacher 
demographic surveys were gathered for children in pre-k settings. In addition, key characteristics of the 
NC Pre-K Program during the 2017-2018 year, along with trends over time (2003-2004 to 2017-2018), 
were examined based on statewide administrative data (NC Pre-K Kids and NC Pre-K Plan). Information 
examined included characteristics of the local NC Pre-K settings, the children served, the qualifications 
of teachers, and distributions and counts of program participants and service providers. 
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Method 
Study Design 
Feasibility Study 


During the previous year (2016-2017), a statewide study of local variations in enrollment and waitlist 
practices was conducted, and these data were used to inform the feasibility and design of the current 
RCT study. The study involved all 91 local NC Pre-K Program contracts (counties or multi-county 
regions), with data gathered through both surveys and phone interviews. The results provided 
information about local variations in implementation of the NC Pre-K Program across four major 
categories of practices: recruitment, application, placement, and waitlist.” These data provided 
information about how children were recruited and placed, which populations of children were served 
by the program, where there were sufficient waitlists to conduct random assignment into the program, 
and the likely make-up of the control group to help guide decision-making related to the feasibility of 
conducting an RCT. The data were reviewed to determine these various conditions related to potential 
sample selection, study recruitment, and the development of procedures for collaborating with local 
Pre-K administrators and programs on the study. Given that NC Pre-K is a mature state pre-k program, it 
was important to consider whether there were local NC Pre-K Programs at the contract level that were 
likely to have sufficiently large applicant lists, enrollment, and waitlists, were not likely to re-assign most 
children on the waitlist to the program or to a similar program, had a relatively low rate of relocation 
out of the county among their families, and were willing to agree to a random assignment process for 
program/waitlist selection for eligible applicants. 


Random Assignment Process 


A small-scale RCT design was used to evaluate the impact of the NC Pre-Kindergarten program in two 
counties. Within each county, children who were eligible for and had applied to the NC Pre-K program 
were randomly assigned into either the treatment (NC Pre-K) or control (waitlist) group. Following 
random assignment and notification of the selection decision, families from both groups were recruited 
for participation in the evaluation study. 


The study used a partially nested, cluster randomized-controlled design. Randomization occurred in 
slightly different ways across the two counties. In county A, randomization was conducted using simple 
random sampling. In county B, randomization occurred within income strata to meet program 
requirements (80% at or below 75% state median income and 20% above). Each county provided the 
researchers with a subset of their lists of eligible applicants for randomization into treatment (NC Pre-K) 
and control (waitlist) groups using a unique identification number. Prior to assignment, the program 
confirmed that all children were eligible for NC Pre-K, based on criteria related to age and family 
income. The number of assignments to the treatment group was made based on the number of slots 
the program needed to fill, with remaining applicants in the pool randomly assigned to the control 


group. 


Participants 
Children/Families 


Randomization was conducted in two local NC Pre-K contracts (counties) which met the criteria based 
on the feasibility study results. Both counties served relatively large numbers of NC Pre-K children (as 
necessary for the study). In one county the program was administered by a public school system and in 
the other by a local Smart Start partnership. The initial randomization pool included a total of 2,243 
children (n=821 county A, n=1,422 county B). Of these, 1,723 were assigned to the treatment condition 
(n=515 county A, n=1,208 county B) and 520 to the control condition (n=306 county A, n= 214 county B). 


Following the randomization process and the notification of the selection decision by the local NC Pre-K 
programs in each county, families were then recruited for participation in the evaluation study. This 
timing and sequence of events resulted in some methodological challenges for sample recruitment. 
Information describing the study, parent permission forms, and parent demographic surveys were 
mailed to families’ homes and also sent via email to families for whom email addresses were available. 
All families were given the option to respond via U.S. mail, email, or online through a dedicated website. 
Additional mailings, follow up emails, and phone calls were carried out, with at least 5 attempts for all 
non-respondents. For children in the random assignment sample who were in classroom-based early 
care and education (ECE) settings, recruitment materials also were sent home by teachers (once they 
agreed to participate) for families who had not yet responded to recruitment requests. 


The study participants included a total sample of 582 children (Treatment=473, Control=109), of whom 
163 children (Treatment=132, Control=31) were Spanish-speaking dual language learners (DLLs). Thus, 
the participation rate from the initial randomization pool was 26% (582/2,243), with slightly higher rates 
for the treatment group (28%) than the control group (21%). 


There was a low rate of crossovers between study conditions —i.e., children who were randomly 
assigned to the treatment group not attending NC Pre-K or children who were randomly assigned to the 
control group attending NC Pre-K. A total of 8 children crossed over between both study conditions, 
including 5 assigned to the control group and 3 to the treatment group. Baseline equivalence analyses 
were conducted to compare characteristics of study participants to non-participants and treatment and 
control groups, and resulting weights were included in the analyses of treatment effects (see analysis 
approach and Appendix 2 for further details). 


The rate of attrition was low overall; 570 children remained in the study from fall to spring during the 
pre-k year and 12 had only fall data. Comparisons of children who remained in the study (i.e., those 
with both fall and spring data) vs those who left the study (i.e., those with only fall data) revealed a few 
significant differences between these two groups for initial skill levels and demographic characteristics. 
Children with fall-only data had higher scores on two measures of literacy skills (WJ Letter Word and 
Passage Comprehension), were slightly older, and had more educated parents. Therefore, the analyses 
of treatment effects adjusted for fall scores as well as included weights based on demographic 
characteristics (see analysis approach for further details). 


Teachers/Classrooms 


Once families gave permission for their children to participate in the study, we contacted the site 
directors or principals of the pre-k programs for children enrolled in classroom-based ECE settings to 
inform them about the study. We then contacted the preschool teachers of study participants to obtain 
their consent to participate. The same recruitment strategies were used for both NC Pre-K and non-NC 
Pre-K classroom-based ECE settings; however, NC Pre-K sites and teachers were required to participate 
in study activities as part of their program requirements, whereas participation for Non-NC Pre-K sites 
and teachers was completely voluntary. A total of 140/156 teachers (90%, Treatment=115, Control=25) 
agreed to participate in the study. These represent the classrooms for 465 (98%) children in the 
treatment group and 27 (25%) children in the control group. 


As seen in Table 1, most characteristics were similar between classrooms attended by children in the 
treatment and control groups. The vast majority of teachers were female (99%); about two-thirds (66%) 
were Black/African-American and about one-quarter (28%) White, with about 7% of Hispanic/Latino 
ethnicity (slightly higher in the control group). Teachers reported an average of about 8-9 years of 
experience teaching pre-k, and an average of about 14-15 years of total teaching experience. The 
average class size was approximately 16 children (about half boys and half girls), and about 70% of the 
children had English as a primary language, about 20% Spanish, and about 10% other languages (with 
higher percentages of English and lower percentages for all other languages for control group 
classrooms). The largest difference was teacher qualifications, which were significantly higher in the 
treatment group. All (100%) of the treatment group teachers held a Bachelor’s degree or higher 
compared to about half (47%) of the control group. The majority (88%) of treatment group teachers had 
a B-K license compared to about one-quarter (26%) of the control group. 


Data Collection Procedures 


Children were assessed at two time points (Fall and Spring) for both the treatment (NC Pre-K) and 
waitlist (control) groups. For children in classroom-based ECE settings (which agreed to participate), 
individual child assessments were conducted by trained data collectors on site at the schools/centers 
and at home for children not in classroom-based settings. All children in the study sample were 
administered the child assessment measures in English. Parallel assessment procedures were used with 
the subsample of Spanish-speaking dual language learners (DLL), with a second administration of the 
same measures in Spanish by a bilingual data collector approximately 2 weeks later. Parents and 
teachers in both groups were asked to complete behavior rating scales following the assessments. An 
initial family demographic survey was gathered at the time of recruitment from the randomization pool 
and a second family survey was gathered at the end of the school year from the study sample. Teachers 
completed demographic surveys online; parents were given the option to complete surveys online or via 
paper forms, in either English or Spanish. For children in pre-k settings, observations of classroom 
practices were conducted with different trained data collectors gathering classroom quality data than 
child assessment data in the same classrooms. All data collectors were trained to specified 
certification/proficiency standards on all measures prior to gathering data. The recommended practices 
by the developers were followed, including didactic training, field practice with feedback, and 
proficiency test-offs. Inter-rater reliability data were collected for 20% of the classroom observations. 


Measures 
Child Assessments 


The child assessment battery consisted of measures appropriate for pre-kindergarteners across five 
primary areas—language, literacy, math, executive function, and behavior skills. (See Table 2 for an 
overview of these measures, including key constructs and scoring.) All of the child assessment measures 
were available in both English and Spanish versions. Most of the measures used in the study were 
norm-referenced, so that for most outcomes, standard scores could be used. These scores take into 
account children’s age, so that the standardized mean score of 100 represents the expected 
performance for an average child at a given age. 


Language and literacy skills were assessed with three subtests from the Woodcock-Johnson Ill Tests of 
Achievement (WJ)" in English and the Bateria II] Woodcock-Mufioz Pruebas de Aprovechamiento (Bat)” 
in Spanish. One measure of language skills was used. The Picture Vocabulary subtest measured 
vocabulary skills, including aspects of both receptive and expressive language. Two measures of literacy 
skills were used. The Letter-Word Identification subtest measured basic pre-reading and reading skills, 
including letter and word recognition and identification skills. The Passage Comprehension subtest 
measured symbolic learning and basic reading comprehension skills. 


Math skills were assessed with two measures from the WJ/Bat. The Applied Problems subtest measured 
math problem-solving skills including simple comparisons, counting, addition, and subtraction. The 
Quantitative Concepts subtest measured knowledge of math concepts, symbols, and vocabulary. 


Executive function was assessed using two measures: Forward Digit Span (FDS) and the Head-Toes- 
Knees-Shoulders Test (HTKS). FDS tests the phonological loop component of working memory. HTKS 
tests children’s inhibitory control, working memory, and cognitive flexibility skills. 


Behavior skills were assessed with two subscales of the Social Skills Improvement System (SSiS) 
completed by teachers and parents. The Social Skills subscale involved ratings of behaviors that 
promote positive interactions while discouraging negative interactions. The Problem Behaviors subscale 
involved ratings of negative behaviors, some commonly occurring and some less commonly, that 
interfere with social skills development. Only the parent ratings were used in the current analyses since 
teacher ratings were not available for control group children not attending classroom-based ECE 
settings. 


In addition, the Pre-IPT Oral (IDEA Proficiency Test) measured oral language proficiency in English for all 
children and also in Spanish for the DLL subsample. Scores on this measure in the fall were examined as 
moderators of treatment effects to determine whether differences in children’s growth on the various 
outcome measures were related to their initial level of language proficiency (1-3= Non-or Limited 
English/Spanish speaker, 4-5= Fluent English/Spanish speaker). Distributions for children in the 
treatment and control groups for the full sample and the DLL subsample are shown in Table 3. 


Teacher Surveys 


Preschool teachers in the study sample were asked to complete electronic surveys about demographic 
and classroom characteristics. The surveys included items about teacher demographic characteristics 
(gender, race, ethnicity), teacher qualifications (education, licensure, teaching experience), and 
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classroom characteristics (class size, proportion of boys/girls, and proportion of children with 
English/Spanish/Other home languages). 


Parent Surveys 


Demographic surveys were gathered from the initial pool of children/families at the time of study 
recruitment. Children’s primary caregivers were asked to complete survey items about children’s 
demographic characteristics (birthdate, gender); Pre-K experience (type of setting, hours in pre-k, name, 
location, and teacher); home language (for child and primary caregiver); and family and household 
characteristics (household composition, caregiver education, family income). A second family survey 
was gathered from the study sample to obtain information about activities with children at home, books 
in the home, beliefs about the child’s kindergarten success, and preschool involvement (if relevant). 


Classroom Observations 


For children in classroom-based ECE settings, the quality of teacher-child instructional interactions was 
measured using the Classroom Assessment Scoring System (CLASS).” The CLASS measures teachers’ 
interactions with children in the areas of social and emotional functioning, classroom organization and 
management, and curriculum implementation to support cognitive and language development. The 
scale includes 10 dimensions organized into three domains, with separate scores calculated for each 
domain. Recent research by the developers has suggested that a single total score, based on the mean 
of the three domain scores, can be used.” For the current study, the total score was used for predictive 
analyses, with information at the domain and dimension levels included in descriptive analyses. The 
scale has demonstrated good interrater reliability based on reported data from the developers (mean 
agreement within one point=87.1%, range=78.8%—-96.9%). For the current study, interrater reliability 
data similarly indicated good agreement within one point based on intraclass correlations (ICC) gathered 
from 19% (n=27) of the observed classrooms (Total=.91, Emotional Support=.87, Classroom 
Organization=.90, Instructional Support=.84). 


State Administrative Data 


Administrative data for the NC Pre-K Program were utilized from three statewide databases—NC Pre-K 
(APP), NC Pre-K Plan (Plan), and NC Pre-K Kids (Kids). Data are entered by system users from all local NC 
Pre-K contracts, each representing a county or multi-county region. APP data consist of the electronic 
application system for NC Pre-K, and include demographic information about children/families, 
household information, eligibility factors, prior placement, assessment evaluation, disabilities, program 
preferences, and placement status. Plan data (updated by contracts as needed) include hierarchically- 
linked information about the contracts (agency contact information), sites (site type, licensing star 
rating, number of classes, and program service dates), classrooms (curriculum, ongoing assessment 
tools, developmental screening tools, daily hours of operation, and class size), and teachers (teacher 
education and licensure/credentials). Kids data (reported by contracts monthly) include hierarchically- 
linked information about the sites (operation days and teacher workdays), classrooms (total monthly 
enrollment and classroom composition—number of NC Pre-K and non-NC Pre-K children), and individual 
children in NC Pre-K (household composition; prior placement; race; ethnicity; gender; birth date; 
primary caregiver’s employment; payment reimbursement rate; attendance; and eligibility factors of 
family income level, limited English proficiency, developmental disability, identified educational need 
and/or IEP, chronic health condition, and parent military service). 
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The NC Pre-K Program Evaluation Team downloaded, verified, corrected, and archived data from these 
systems monthly. Data from APP were used to examine demographic and family variables associated 
with determining eligibility to NC Pre-K for the entire randomization pool for the RCT study. Data from 
Plan and Kids were used to examine statewide program characteristics for NC Pre-K from 2003-2004 
through 2017-2018 (July 1—June 30), focusing on the most recent year, along with comparisons of some 
key characteristics over time. 


Analysis Approach 


Although most data were available on most measures for most participants, some data were missing 
due to non-response. For children in the randomization pool, demographic and family variables from NC 
Pre-K APP data were available for most children. Any missing data were multiply imputed using the fully 
conditional specification method (FCS) since many of the variables at the level of randomization were 
binary. For children in the study sample with missing data or who were lost to follow-up over the course 
of the study, we used Markov chain Monte Carlo methods to multiply impute missing data for children. 


Weighting to Account for Potential Bias 


The design of this study lent itself to two sources of potential bias: a) confounding bias due to lack of 
baseline equivalence after randomization (i.e., exchangeability between treatment and control 
children); and b) selection bias due to non-participation. Standardized mean differences and probability 
differences were used to identify imbalance in demographic characteristics for the two different sources 
of bias in the study. To account for these two types of bias, we constructed inverse probability of 
treatment and censoring weights (IPTCW). 


Broadly, inverse probability weights (IPW) are a statistical tool that allows analyses to be adjusted for 
confounding and selection bias. IPW adjust for these biases by parametrically standardizing the study 
population to represent a pseudo-population. Inverse probability of treatment weights (IPTW) address 
confounding due to lack of exchangeability (i.e., baseline equivalence between participant treatment 
and control groups). “" Inverse probability of censoring weights (IPCW) assume censoring (i.e., 
participation) can be predicted by measured covariates such that weights allow participation to be 
independent of those variables.” Inverse probability of treatment and censoring weights (IPTCW) are 
formed by multiplying treatment and censoring weights. What that means is participants are weighted 
so that their treatment and participation status are independent of measured confounding covariates.” 
This weighted analysis approach allows the effect of treatment assignment on outcomes to be 
independent of confounding and selection bias. The contrast we estimated with this approach 
compares the causal effect of the NC Pre-K program on child outcomes if everyone had been exposed to 
NC Pre-K and everyone who was randomized had participated compared to the effect on child outcomes 
if none of the children had been exposed to NC Pre-K. (See Appendix 2 for further technical details 
about the construction of the weights and the baseline equivalence results.) 


Statistical Comparisons 


To evaluate the effect of exposure to the NC Pre-K Program on preschool-aged children, we conducted 
intent-to-treat (ITT) analyses. To accommodate the nesting of children within schools, two-level HLM 
analyses were used to compare outcomes for children randomly assigned to the treatment (NC Pre-K) or 
the control (waitlist) groups. Partially nested models were fit to account for the effect of school-level 
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clustering for children assigned to NCPK.”' These models also allowed for separate variances to be 
estimated for treatment and control children. The models specified random intercepts for treatment 
and control groups and estimated an ICC for each treatment school that two or more treatment children 
attended while holding the ICC among the control group children and treatment children who were the 
only child at their school in the study to be zero. County was included as a fixed effect because 
randomization took place at the county level. An attempt was made to include county as a block ina 
blocked partially nested design but this resulted in model over specification. Children’s fall scores on 
the outcome measure also were included to account for potential pre-existing differences and to 
increase the precision of the treatment impact estimates. The product of the two IPWs were applied as 
weights in all analyses to account for potential bias due to differences between the sample and 
participation. Multiple imputations were conducted to account for missing data on the child outcomes 
(n=20 multiple imputed data sets). 


Two models were used to evaluate the impact of NC Pre-K on children’s academic and social skills at the 
end of the Pre-K year. (See Appendix 2 for model specifications.) The first model was intended to 
determine the most direct impact of treatment with only treatment condition, county and previous fall 
scores included as predictors. The use of fall scores as a covariate allows for the estimate of residualized 
gains in children’s spring scores. In addition to the main outcome analyses, we also conducted limited 
moderation analyses that built upon the first model with the inclusion of two moderators. These 
moderation analyses examined whether NC Pre-K differentially impacted children under certain 
conditions, specifically oral language proficiency (measured by the pre-IPT in English for English 
outcomes and in Spanish for Spanish outcomes) and classroom quality (indicated by CLASS total score). 
Language proficiency scores and CLASS total scores were entered into the model independently and 
then crossed with treatment to determine if there were any interactions with treatment status. 
Because many children in the control group were not attending a pre-k program, we created an 
indicator variable and used the imputed centered mean for classroom measures (e.g., CLASS=0) for 
those not attending pre-k. To account for testing multiple outcomes, Benjamini-Hochberg adjustments 
were applied. Effect sizes were calculated for significant effects as the coefficient divided by the pooled 
standard deviation (B /SD). 


To test the sensitivity of the intent-to-treat (ITT) analyses to bias due to condition crossovers, we 
conducted treatment-on-the treated (TOT) analyses. The analytic strategy for the TOT was identical to 
that used in the ITT, with the use of partially nested HLM to accommodate the structure of the nested 
data and two models, an initial model evaluating the impact of treatment and a second model including 
potential moderators of treatment. The primary difference between the TOT and the ITT analyses was 
that for the TOT analyses, children who were assigned to the control group but ended up receiving the 
treatment were reassigned to the treatment group and children who were assigned to the treatment 
group but did not receive it were reassigned to the control group. Results from the ITT and TOT 
analyses could then be compared to evaluate the impact of who was assigned to the treatment 
condition versus who actually received it. The same sets of analyses were conducted for the full sample, 
the DLL subsample English outcomes, and the DLL subsample Spanish outcomes. 
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Results 
Descriptive Analyses 


Information from family surveys were examined descriptively to provide additional information about 
child and family characteristics for the study sample. The home environments for the treatment and 
control groups were fairly similar across a number of dimensions. There were no differences in the 
highest level of parent education, with about one-third in each group having a High School diploma or 
less (Treatment=34.6%, Control=34.0%), Some college (Treatment=35.2%, Control=34.0%), or a 
Bachelor’s degree or above (Treatment=30.3%, Control=32.0%). Families reported having an average of 
around 49 books in their homes, with similar numbers reported by each group, although there was 
substantial variability within groups [Treatment= 47.7 (SD=55.9), Control=51.7 (SD=53.5)]. Families were 
asked to report the frequency of various activities at home with their children (e.g., reading, telling 
stories, singing songs, counting, etc.) on a 4-point scale (not at all, 1-2 times/week, 3-6 times/week, 
every day). The average scores were identical between the two groups (M=2.9, SD=0.6), indicating that 
families engaged in these activities with their children an average of nearly 3-6 times/week. 


Similarly, when families were asked to report the frequency of parent involvement activities with their 
child’s preschool, the overall scores were nearly identical for both groups [Treatment=3.2 (SD=0.9), 
Control=3.3 (SD=1.1)]. In order to better understand the comparative context of the control group, the 
researchers asked families to report the type of settings for children in ECE. Just under half (n=47, 
45.6%) reported that they did not use any non-parental ECE. Private preschool was reported by 29.1% 
(n=30); public pre-k settings, including Head Start, by 7.8% (n=8); family child homes by 3.9% (n=4); care 
in the child’s home by 8.7% (n=9); and for the remainder, the type was either unknown (4.9%, n=5) or 
not reported (5.5%, n=6). 


The various outcome measures of children’s language, literacy, math, executive function, and social 
skills were examined descriptively for the treatment and control groups. Children’s fall (beginning of the 
pre-k year) and spring (end of the pre-k year) scores were analyzed for the full sample on English 
measures and the DLL subsample on both English and Spanish measures. (See Table 4, Table 5, and 
Table 6.) For the full sample, children in both groups were scoring close to the mean on norm- 
referenced measures at both time points. For the DLL subsample, scores tended to be lower for skills 
measured in Spanish than in English. 


Observational data measuring the quality of classroom practices also were examined descriptively. 
Average CLASS total, domain, and dimension scores were calculated for the observed classrooms of 
children in the treatment and control groups attending pre-k settings. (See Table 7.) Not surprisingly, 
there were more classrooms represented in the treatment group than in the control group. Scores were 
higher in the treatment group and were slightly higher than in many samples. However, the pattern was 
similar for both groups (and similar to the typical pattern for ECE classrooms), with scores in the medium 
range in most areas (Total, Emotional Support, Classroom Organization) and in the low range for one 
area (Instructional Support). 


Treatment Effects — Full Sample 


The effects of NC Pre-K on outcomes in the areas of language, literacy, math, executive function, and 
social skills were examined for children in the full sample. Analyses compared differences between the 
treatment and control groups for outcomes during pre-k. In addition, children’s oral language 
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proficiency and pre-k classroom quality were examined as potential moderators of the effects of 
treatment. Two sets of analyses were conducted — the ITT analyses provided the primary set of results 
testing the impact of NC Pre-K based on random assignment to treatment condition and the TOT 
analyses provided comparison results based on actual treatment received (accounting for crossovers). 


Intent-to-Treat (ITT) — Full Sample 


Two significant differences between the treatment and control groups were found based on the ITT 
results for the full sample (as indicated by significant coefficients for Treatment). (See Table 8 and 
Figure 1 and Figure 2.) There were positive effects of treatment on residualized gains for vocabulary 
skills (WJ Picture Vocabulary B=2.64, effect size=.22) and on letter and word recognition skills (WJ Letter- 
Word Identification, B=3.90, effect size=.27). There were no moderating effects of children’s English 
proficiency level on treatment for any of the outcome measures. There also were no moderating effects 
of pre-k classroom quality (CLASS scores) on treatment for children who attended preschool settings. 


These results indicate that eligible children who were randomly assigned to NC Pre-K (treatment group) 
had better vocabulary and letter and word recognition skills at the end of pre-k (after adjusting for their 
skills at the beginning of pre-k) compared to children who were randomly assigned to the waitlist 
(control group). Furthermore, these differences were not differentially affected by children’s level of 
English language proficiency or the quality of their pre-k classrooms. 


Treatment-on-Treated (TOT) — Full Sample 


Next, parallel analyses were conducted to address condition crossovers — a few children in the 
treatment group did not attend NC Pre-K and a few children in the control group did attend NC Pre-K. 
The results from the TOT analyses were consistent with the results from the ITT analyses for the full 
sample, with two significant differences between the treatment and control groups (as indicated by 
significant coefficients for Treatment). (See Table 9.) There were positive effects of treatment on 
residualized gains for vocabulary skills (WJ Picture Vocabulary, B=3.24, effect size=.27) and on letter and 
word recognition skills (WJ Letter-Word Identification, B=4.03, effect size=.28). There were no 
moderating effects of children’s English proficiency level on treatment for any of the outcome measures. 
There also were no moderating effects of pre-k classroom quality (CLASS scores) on treatment for 
children who attended preschool settings. 


These results indicate that eligible children who attended NC Pre-K had better vocabulary and letter and 
word recognition skills at the end of pre-k (after adjusting for their skills at the beginning of pre-k) 
compared to children did not attend NC Pre-K. Furthermore, these differences were not differentially 
affected by children’s level of English language proficiency or the quality of their pre-k classrooms. 
These findings from the TOT analyses are consistent with those from the ITT analyses. The similarity of 
the two sets of results indicates that any differences among families that may have affected decisions to 
enroll in NC Pre-K did not have any significant impact on the effects of treatment. Thus, these findings 
suggest that the ITT results are robust to potential threats from selection bias due to variability in 
treatment uptake by those initially offered NC Pre-K through random assignment and to exposure to 
treatment by those in the control group. 


Treatment Effects - DLL Subsample 
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The effects of NC Pre-K on outcomes in the areas of language, literacy, math, executive function, and 
social skills were examined for children in the DLL subsample. Analyses compared differences between 
the treatment and control groups for outcomes in both English and Spanish during pre-k. In addition, 
children’s oral language proficiency and pre-k classroom quality were examined as potential moderators 
of the effects of treatment. These analyses provided an additional set of results to specifically examine 
the outcomes for this subsample of children. Two sets of analyses were conducted to examine the 
effects for the DLL subsample — the ITT comparisons provided the primary set of results to test the 
impact of NC Pre-K based on random assignment to treatment and the TOT comparisons provided 
results of sensitivity analyses to test the impact of NC Pre-K based on actual treatment received. 


ITT - DLL Subsample 


For skill measured in English, two significant differences between the treatment and control groups 
were found based on the ITT results for the DLL subsample (as indicated by significant coefficients for 
Treatment). (See Table 10 and Figure 3 and Figure 4.) There were positive effects of treatment on 
residualized gains for letter and word recognition skills (WJ Letter-Word Identification, B=7.93, effect 
size=.64) and knowledge of math concepts (WJ Quantitative Concepts, B=5.27, effect size=.39). For two 
additional measures, vocabulary (WJ Picture Vocabulary) and math problem-solving (WJ Applied 
Problems), significant effects were not maintained after applying the adjustments for multiple tests. 
There were no moderating effects of children’s English proficiency level on treatment for any of the 
outcome measures. There also were no moderating effects of pre-k classroom quality (CLASS scores) on 
treatment for children who attended preschool settings. 


These results indicate that among the DLL subsample, eligible children who were randomly assigned to 
NC Pre-K (treatment group) had better letter and word recognition skills and knowledge of math 
concepts at the end of pre-k (after adjusting for their skills at the beginning of pre-k) compared to 
children who were randomly assigned to the waitlist (control group). Furthermore, these differences 
were not differentially affected by children’s level of English language proficiency or the quality of their 
pre-k classrooms. 


For these same skills measured in Spanish, the ITT analyses found no significant differences between the 
treatment and control groups for children in the DLL subsample. (See Table 11.) There was one 

marginally significant effect for written comprehension skills (Bat PC), but this effect did not persist after 
applying the adjustments for multiple tests. There also were no moderating effects of children’s Spanish 
proficiency level or pre-k classroom quality (for children who attended preschool settings) on treatment. 


TOT - DLL Subsample 


Similar results for the TOT analyses and the ITT analyses were found for the DLL subsample for skills 
measured in English. Three significant differences between the treatment and control groups (as 
indicated by significant coefficients for Treatment) were found for the TOT analyses. (See Table 12.) 
There were positive effects of treatment on residualized gains for vocabulary skills (WJ Picture 
Vocabulary, B=5.56, effect size=.42), on letter and word recognition skills (WJ Letter-Word Identification, 
B=8.81, effect size=.71), and on knowledge of math concepts (WJ quantitative concepts, B=5.61, effect 
size=.43). Similarly to the ITT results, significant effects for math problem-solving (WJ AP) were not 
maintained after applying the adjustments for multiple tests, as well as for one measure of executive 
function skills (HTKS). There were no moderating effects of children’s English proficiency level on 
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treatment for any of the outcome measures. There also were no moderating effects of pre-k classroom 
quality (CLASS scores) on treatment for children who attended preschool settings. 


These results indicate that among children in the DLL subsample, eligible children who attended NC Pre- 
K had better vocabulary, letter and word recognition skills, and knowledge of math concepts at the end 
of pre-k (after adjusting for their skills at the beginning of pre-k) compared to children who did not 
attend NC Pre-K. Furthermore, these differences were not differentially affected by children’s level of 
English language proficiency or the quality of their pre-k classrooms. The TOT results paralleled the 
findings from the ITT analyses for both measures (math concepts and letter/word recognition). Fora 
third measure (vocabulary), only the TOT results were significant. However, the coefficient for 
vocabulary in the ITT analyses was statistically significant (p<.01, effect size=.33) before applying the 
adjustments for multiple tests. 


For these same skills measured in Spanish, the TOT analyses found no significant differences between 
the treatment and control groups for children in the DLL subsample. (See Table 13.) Similarly to the 
results for the ITT analyses, there was one significant effect for written comprehension skills (Bat PC), 
but this effect did not persist after applying the adjustments for multiple tests. There also were no 
moderating effects of children’s Spanish proficiency level or pre-k classroom quality (for children who 
attended preschool settings) on treatment. These findings are consistent between the ITT and TOT 
analyses. 


Program Characteristics 


Descriptive data were analyzed to provide information about the NC Pre-K Program for the 2017-2018 
school year. In 2017-2018, the NC Pre-K Program served 30,035 children in 2,007 classrooms located in 
1,167 sites. The average total class size was approximately 16 children, with an average of 86% (13.5) of 
those children funded by NC Pre-K. On average, children attended NC Pre-K for 137 days, which 
represents 81% of the average days of operation (170) or 76% of the 180 intended days of operation 
based on program guidelines. In terms of NC star-rated licenses (the Quality Rating and Improvement 
System or QRIS in NC), nearly 80% of the sites had a five-star rating and 15% had a four-star rating, with 
the remainder either temporary or in process. (See Table 14.) 


Almost all classrooms reported using an early childhood curriculum, ongoing assessment tool, and 
developmental screening tool from the approved lists provided by the NC Pre-K Program Guidelines. 
Over 90% of classrooms reported using Creative Curriculum and its companion assessment (Teaching 
Strategies Gold or the Creative Curriculum Developmental Continuum). All classrooms reported using 
an approved developmental screening tool, with most using either DIAL (53%) or Brigance (39%). (See 
Table 15.) NC Pre-K classrooms were located in approximately half (52%) public school settings; about 
one-third (33%) private settings (25% for-profit and 8% non-profit child care centers); and 15% Head 
Start (4% administered by public schools and 11% not). (See Table 16.) 


In 2017-2018, the children served in NC Pre-K were about half boys and half girls from a variety of racial 
and ethnic backgrounds, including slightly under one-half (48%) White, over one-third (36%) 
Black/African-American, and almost one-quarter (24%) of Hispanic/Latino ethnicity. Nearly 80% of the 
children attending NC Pre-K had at least one parent in the workforce. (See Table 17.) Children served 
by the NC Pre-K Program primarily came from low-income families, with 88% eligible for free or 
reduced-price lunch. Children also varied on other eligibility factors, ranging from 19-22% with limited 
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English proficiency or a developmental/educational need to 4-5% with an identified disability, chronic 
health condition, or military parent. (See Table 18.) Information on children’s prior placement indicated 
that almost three-quarters had never previously been served in any preschool setting (58%) or were 
currently unserved (14%) at the time of enrollment (see Table 19). 


Almost all (over 99%) lead teachers in the NC Pre-K Program in 2017-2018 had at least a bachelor’s 
degree in both public school and private settings (see Table 20). Nearly all teachers in public school 
settings (96%) and most in private settings (86%) had a Birth-Kindergarten (B-K) license (or the 
equivalent). Relatively few teachers in public school settings (0.4%) and in private settings (11%) 
reported no credential. (See Table 21.) 


Trend Analyses 


Results from trend analyses examined whether there have been any long-term changes in key program 
characteristics since the NC Pre-K Program (formerly More at Four) became statewide (2003-2004) 
through the 2017-2018 program year. Evidence of a linear trend, or change over time in a given 
characteristic, is indicated by R* > .70; conversely, evidence of no linear trend, or little change over time, 
is indicated by R*< .70. Distributions are shown for key program characteristics over time, including 
setting type (see Table 22), children’s prior placement (see Table 23), teacher education (see Table 24), 
and teacher licensure/credentials (see Table 25). 


Results indicated that there was little change over time in the distribution of NC Pre-K classrooms by 
setting types (percentages of public pre-k, private, and Head Start), with no evidence of linear trends for 
any of these categories (as indicated by R*< .70). (See Figure 5.) The results for children’s prior 
placement similarly showed fairly consistent patterns over time, with no evidence of linear trends for 
the proportion of children never served (never served) and the proportion not served at the time of 
enrollment (unserved). (See Figure 6.) 


Results from trend analyses indicated that one consistent change in the program has been the increases 
in teacher qualifications (education and licensure/credentials) over time. There were significant 
changes over time for all three aspects of teacher qualifications that were examined (see Figure 7). For 
teacher education (percentage with bachelor’s degree or above), results indicate an increasing trend 
over time (R? = 0.78). It should be noted that teacher education has essentially reached the maximum 
level from cohorts 9-13, which decreases the goodness-of-fit statistic, although it is still within the 
acceptable range. For lead teacher licensure and credentials, the results indicate two parallel trends — 
an increasing trend in the percentage of those with a B-K license (R* = 0.98) and a decreasing trend in 
the percentage of those with no credential (R? = 0.79). 
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Conclusions 


The current study addressed the issue of the benefits of offering children and families access to 
enrollment in NC Pre-K — the state pre-k program in North Carolina. Whether children benefit from 
participation in NC Pre-K was tested through a small-scale RCT study conducted in two counties. Such a 
design is considered the gold standard for examining this type of question. 


Children who were eligible for and had applied to NC Pre-K were randomly assigned to either the 
program (treatment group) or the waitlist (control group). Results from the study indicated some 
consistent effects on language and literacy skills at the end of pre-k, with better performance for 
children in the treatment group than in the control group (after adjusting for initial performance). 
Differences were found for vocabulary and letter and word recognition skills — two key measures related 
to subsequent school readiness as well as later reading and school success. For the subsample of 
Spanish-speaking DLLs, these effects again were found for letter and word recognition skills measured in 
English, as well as for knowledge of math concepts in English (a skill that has both math and literacy 
aspects). Moreover, all of these results were found in both the ITT and the TOT analyses, suggesting 
that these effects are consistent when considering actual treatment received rather than randomly 
assigned. 


However, it also is important to note that the study did not find significant effects for other measures 
after adjusting for multiple tests, including other measures of literacy (written comprehension) and 
math skills (problem solving), executive function, and parent ratings of social skills and problem 
behaviors. The researchers used strict criteria for significance as well as conservative adjustments to 
guard against false positive effects, and thus excluded some results from the final set of findings that did 
not meet these criteria. There also were no significant differences when skills were measured in Spanish 
rather than in English for the DLL subsample, although that is less surprising given that the language of 
instruction in these classrooms generally is in English. We also asked, secondarily, whether children’s 
level of oral language proficiency or the quality of their pre-k classroom (for children in pre-k settings) 
affected the impact of treatment. The results indicated that treatment was not differentially affected by 
either of these factors. 


Although the study had few cases of children not receiving the treatment condition they were assigned 
(crossovers) or not remaining in the study for the duration (attrition), the analysis approach utilized 
methods to address any resulting differences in baseline equivalence. However, there were 
methodological challenges affecting sample size that may have implications for the limited set of 
findings as well as the generalizability of the results. First, given that NC Pre-K is a mature state pre-k 
program, a feasibility study was conducted to determine viable locations for conducting the study. 
Based on these results, the study was limited to the selection of two counties that met the criteria with 
regard to program enrollment, waitlist, and ability to utilize a random assignment selection process. 
Second, children and families were recruited for study participation following randomization and 
notification of the selection decision by the program, which may have reduced the study participation 
rate, even slightly more for the control group. 


The limited set of positive findings with regard to treatment effects may be partially explained by the 
settings in which the study was conducted. North Carolina in general has a long-standing history of 
providing ECE and family supports that are particularly directed toward low-income families such as 
those applying to the NC Pre-K Program. Even though the majority of children in the control group were 
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not in a pre-k program, they still may have benefited from services provided through initiatives such as 
Smart Start that would have positive impacts on the types of outcomes measured in the current study. 
Further, the small-scale design of the study, along with the necessary conditions for feasibility, imposed 
limitations on the locales within which it could be conducted. Because the study included only two 
counties, both of which were well-resourced, families in the control group likely had other opportunities 
for educational and social supports outside of participation in a pre-k program. 


Results from previous evaluations of the NC Pre-K Program generally have found a wider range of 
positive effects of pre-k participation on children’s school readiness and early elementary academic and 
social outcomes using other types of comparison designs.*" *" *” However, these studies have included 
more representative samples with greater power to detect differences. Similar results have been found 
for other studies of state pre-k programs using similar methodologies. *” ” A recent statewide study of 
the TN VPK Program also using an RCT design with a larger sample found significant positive effects on 
children’s academic skills through pre-k, but these effects did not last as they followed them from 
kindergarten through third grade. 


In sum, the results from this small-scale RCT study suggest that there is some evidence of positive 
effects for participation in the NC Pre-K Program, particularly around language and literacy knowledge, 
as well as literacy and math concepts for DLLs. However, there are limitations to the current study and 
further research would be beneficial. State pre-k programs are widespread throughout the U.S., and 
there is a range of research evidence to support the benefits of ECE programs. *” However, there still 
are unanswered questions about the best ways to support and sustain young children’s learning and 
education in pre-k and beyond in order to meet the goal of helping all children achieve success in school. 
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Table 1. Characteristics of Sample Teachers and Classrooms 


All Control Treatment 
Teacher Characteristics (n=140) (n=25) (n=115) 
n (%) n (%) n (%) 
Gender 
Female 121 (99.2) 19 (100) 102 (99.0) 
Male 1 (0.8) 0 1 (1.0) 
Race 
Black/African-American 80 (66.1) 11 (61.1) 69 (67.0) 
Other 7 (5.8) 1 (5.6) 6 (5.8) 
White 34 (28.1) 6 (33.3) 28 (27.2) 
Hispanic/Latino ethnicity 8 (6.6) 2 (10.5) 6 (5.8) 
Education 
HS/AS 10 (8.5) 10 (52.6) 0 
BA/BS 94 (79.7) 4 (21.1) 90 (90.9) 
MA/MS 14 (11.9) 5 (26.3) 9 (9.1) 
Licensure 
Birth to Kindergarten 96 (78.7) 5 (26.3) 91 (88.3) 
Other state/NC license 5 (4.1) 1 (5.3) 4 (3.9) 
None 21 (17.2) 13 (68.4) 8 (7.8) 
Teacher Type 
Non-NC Pre-K 18 (14.8) 17 (89.5) 1 (1.0) 
NC Pre-K 104 (85.2) 2 (10.5) 102 (99.0) 
Teacher Experience 
Years teaching pre-k 
Mean (SD) 9.0 (6.7) 8.0 (7.3) 9.2 (6.6) 
Min-Max 0.0 (33.8) 0.0 (25.0) 0.0 (33.8) 
Years teaching any age 
Mean (SD) 14.6 (8.2) 13.3(7.0) 14.8 (8.3) 
Min-Max 0.0 (40.8) 1.0(28.0) 0.0 (40.8) 
Classroom Characteristics | Mean (SD) Mean (SD) Mean (SD) 
Class size 16.3 (3.1) 14.7 (5.1) 16.6 (2.5) 
Proportion of boys 0.5 (0.1) 0.5 (0.2) 0.5 (0.1) 
Proportion of children’s home languages 
English 0.7 (0.2) 0.9 (0.2) 0.7 (0.2) 
Spanish 0.2 (0.2) 0.1 (0.2) 0.2 (0.2) 
Other 0.1 (0.1) 0.0 (0.1) 0.1 (0.1) 
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Table 2. Child Outcome and Classroom Quality Measures 


Measure 


Scoring 


Language Skills 
Vocabulary 
WJ Picture Vocabulary (Subtest 14) / Bat Vocabulario sobre Dibujos (Prueba 14) 


Standard score 
Mean=100, SD=15 


Literacy Skills 
Letter-and Word Recognition 
WJ Letter-Word Identification (Subtest 1) / Bat Identificaci6n de Letras y Palabras (Prueba 
1) 


Written Comprehension 
WJ Passage Comprehension (Subtest 9) / Bat Comprension de Textos (Prueba 9) 


Standard score 
Mean=100, SD=15 


Standard score 
Mean=100, SD=15 


Math Skills 


Math Problem-Solving 
WJ Applied Problems (Subtest 10) / Bat Problemas Aplicados (Prueba 10) 


Math Concepts and Symbols 
WJ Quantitative Concepts (Subtest 18) / Bat Razonamiento Cuantitativo (Prueba 18) 


Standard score 
Mean=100, SD=15 


Standard score 
Mean=100, SD=15 


Executive Function 
Working Memory 
Forward Digit Span (English/Spanish) 


Executive Function 
Head-Toes-Knees-Shoulders Test (HTKS) (English/Spanish) 
Behavior Skills 
Social Skills 
SSiS Social Skills subscale 


Problem Behaviors 
SSiS Problem Behaviors subscale 


Raw score 
Range=1-8 


Raw score 
Range=0-60 


Standard score 
Mean=100, SD=15 


Standard score 
Mean=100, SD=15 


Classroom Quality 
Teacher-Child Instructional Interactions 


CLASS Total & Emotional Support, Classroom Observation, & Instructional Support Domains 


Total and Domain 
scores range=1.0—7.0 


WJ = Woodcock III Johnson Tests of Achievement; Bat = Bateria I!| Woodcock-Mufioz Pruebas de Aprovechamiento; SSiS = Social Skills 


Improvement System; CLASS = Classroom Assessment Scoring System. 
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Table 3. Child Language Proficiency Levels 


Full Sample 


DLL Subsample 


Pre-IPT 
Proficiency Level 


Level 1 (Non-Speaking) 
Level 2 (Limited) 

Level 3 (Limited) 

Level 4 (Fluent) 

Level 5 (Fluent) 


Total 


English Proficiency 


All Control 

% n % n 
7.0 41 9.0 9 
15.8 90 12.0 12 
15.1 86 19.0 19 
18.6 106 16.0 16 
43.4 247 440 44 
100.0 582 100.0 109 


Treatment 
% n 
6.6 31 
16.6 78 
14.3 67 
19.2 90 
43.3 203 
100.0 473 


English Proficiency 


Spanish Proficiency 


All Control Treatment Control Treatment 
% n % n % n % n % n 

16.7 27 30.0 9 13.6 18 10.0 3 19.8 26 
24.1 39 20.0 6 25.0 33 40.0 12 27.5 36 
16.7 27 16.7 5 16.7. 22 20.0 6 20.6 27 
18.5 30 10.0 3 20.5 27 10.0 3 145 19 
24.1 39 23.3 7 24.2 32 20.0 6 17.6 23 
100.0 163 100.0 31 100.0 132 100.0 31 100.0 132 
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Table 4. Outcome Scores for Full Sample 


All Control Treatment 


N Mean SD Min Max N Mean SD Min Max N Mean SD Min Max 


Language 

WJ Picture fall 560 96.41 13.30 42 133 98 97.08 14.52 48 133 462 96.27 13.04 42 132 
Vocab 

ne spring 556 96.74 11.97 44 137 100 95.94 13.41 52 126 456 96.92 1164 44 137 
Literacy 

WJ Letter- fall 569 100.2 14.32 64 178 100 100.8 17.21 66 178 469 100.1 13.64 64 159 
Word ID 

spring 561 101.6 14.67 61 176 103 99.49 18.78 61 161 458 102.1 13.55 68 176 
WJ fall 568 98.90 9.56 71 138 100 99.36 10.87 72 138 468 98.81 9.26 71 138 
Passage hein 

Comp ee 560 100.2 9.78 68 144 103 102.3 11.18 73 144 457 99.70 9.38 68 138 
Math 

WSJ fall 569 102.0 12.56 51 149 100 102.4 13.57 58 149 469 101.9 12.35 51 136 
Applied eee 

Problems pe 561 103.5 11.45 53 150 103 103.1 14.29 53 150 458 103.6 10.72 55 135 
WJ Quant fall 553 95.43 13.34 68 149 97 96.71 15.94 73 149 456 95.16 12.72 68 137 
Concepts 


spring 557 96.50 13.81 61 145 101 96.97 16.12 63 145 456 96.40 13.26 61 134 


Executive Function 


Forward fall 569 3.89 0.95 1 70 100 3.88 0.88 1 5 469 3.90 0.97 1 7 
Digit Span ; 
spring 561 4.20 0.98 1 80 103 4.20 1.11 1 8 458 4.20 0.95 1 8 
HTKS fall 568 11.72 15.25 QO 550 100 10.22 14.09 0 46 468 12.04 15.48 0 55 
spring 559 21.59 18.28 QO 600 103 20.17 17.32 0 55 456 21.91 18.50 0 60 
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Table 4 (Cont.) Outcome Scores for Full Sample 


Behavior Skills 


SSiS Social fall 
Skills 


spring 
SSiS fall 
Problem nite 
Behaviors ne? 


375 


432 


Mean 


107.6 
110.5 


98.22 


97.88 


All 


Std 


12.93 
14.08 


14.93 


15.67 


Min 


67 
51 


77 


77 


Max 


134 


134 


160 


160 


N 


84 


84 


81 


84 


Mean 


105.9 


108.8 


100.2 


101.4 


Control 


Std 


12.30 
13.66 


16.54 


17.94 


Min 


67 


75 


77 


77 


Max 


134 


134 


160 


160 


N 


295 
348 


294 


348 


Mean 


108.1 


110.9 


97.69 


97.03 


Treatment 
Std Min 
13.08 71 
14.17 51 
14.44 77 
14.97 77 


Max 


134 


134 


156 


160 
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Table 5. Outcome Scores for DLL Subsample — English Measures 


All Control Treatment 
N Mean _ Std Min Max N_- Mean _ Std Min Max N_ Mean _ Std Min Max 
Language 
WJ Picture fall 155 86.59 15.94 42 117. 28 «284.21 17.41 48 112 127 87.12 15.62 42 117 
Vocab 
oe spring 157 8835 13.43 44 115 28 85.14 16.95 52 110 129 89.05 12.51 44 115 
Literacy 
WJ Letter- fall 162 94.33 11.94 64 119 30 88.93 9.56 66 113 132 95.56 12.12 64 119 
Word ID 
be spring 161 95.40 13.31 61 133 31 86.06 13.82 61 133 130 97.63 12.22 69 130 
WJ fall 162 98.93 8.01 71 116 30 98.93 8.32 76 115 132 98.92 7.97 71 116 
Passage Sarin 
Comp eng 161 100.1 7.55 82 133 31 100.1 9.66 82 133 130 100.1 7.00 86 120 
Math 
WJ fall 162 97.88 13.30 58 128 30 94.73 13.97 58 118 132 98.59 13.09 62 128 
Applied eer 
Problems ping 161 100.4 12.14 53 129 31 93.84 14.90 53 120 130 102.0 10.87 76 129 
WJ Quant fall 152 91.42 11.53 68 121 27 90.00 11.11 74 121 125 91.73 11.64 68 120 
Concepts , 
spring 159 92.92 13.28 61 122 29 88.28 11.63 63 111 130 93.95 13.44 61 122 
Executive Function 
Forward fall 162 3.55 0.87 1 6 30 3.60 1.00 1 5 132 3.54 0.84 1 6 
Digit Span ; 
spring 161 3.83 0.77 1 6 31 3.74 0.73 3 6 130 3.85 0.78 1 6 
HTKS fall 162 11.33 15.98 0 55 30 8.60 13.07 0 43 132 11.95 16.55 0 55 
spring 161 20.84 18.79 0 60 31 15.55 16.81 0 54 130 22.11 19.07 0 60 
Behavior Skills 
SSiS Social fall 102 107.6 11.97 67 134 26 104.7 12.22 67 120 76 108.5 11.81 83 134 
Skill 
~ spring 102 109.5 13.26 75 133 24 106.0 13.90 75 126 78 110.5 12.97 82 133 
SSiS fall 98 101.3 11.71 77 134 23 100.6 10.39 84 122. 75 101.6 12.13 77 134 
Problem ‘ans 
Behaviors Pine 101 102.7 13.80 77 160 23 105.8 18.23 78 160 78 101.8 12.20 77 142 
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Table 6. Outcome Scores for DLL Subsample — Spanish Measures 


All Control Treatment 
N Mean _ Std Min Max N Mean _ Std Min Max N Mean _ Std Min Max 
Language 
Bat Picture fall 148 74.63 18.75 41 114 29 7352 2177 43 110 119 7490 1803 41 114 
wogab spring 143 73.54 18.72 41 113 29 71.17 2004 41 «110 114 7414 1842 42 113 
Literacy 
Bat Letter- fall 161 90.76 9.72 67 119 30 8687 1037 69 107 131 9166 938 67 119 
Word ID coring 161 «89.41 1027 67 117 31 8639 9.77 69 107 130 9013 1029 67 117 
Bat fall 152 87.37 1685 47 120 27 82.63 15.73 48 115 125 8839 1696 47 120 
Passage : 
spring 
Comp 157 90.59 1669 46 118 30 87.60 1659 48 110 127 9130 1670 46 118 
Math 
Bat fall 161 94.71 1133 57 125 30 9440 939 77 113 131 94.79 1176 57 125 
Applied ; 
spring 
Problems 161 96.25 1230 52 131 31 9439 843 80 112 130 96.70 1304 52 4131 
Bat Quant fall 159 86.83 10.25 64 118 30 82.90 987 68 113 129 87.74 1016 64 118 
Concepts soring 160 86.04 11.16 59 128 31 83.23 9.17 64 100 129 86.72 1151 59 128 
Executive Function 
Forward _ fall 161 3.52 0.81 1 6 30° 337 +093 1 5 13t. BSR “A079 1 6 
Digit Span : 
: spring 
- Spanish 160 3.60 0.84 1 6 30 3.50 0.82 2 6130. 2362» 7085 1 5 
HTKS - fall 161 14.78 18.18 OG: 59 30. 4733¢ 1627 0 52 131 15.34 18.60 0 59 
Spanish soring 160 25.36 20.78 Oo 59 31 22.45 21.32 0 58 129 26.05 20.67 0 59 
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Table 7. Classroom Quality Scores 


Total Control Treatment 
(n=139) (n=25) (n=114) 

Mean (SD) Mean (SD) Mean (SD) 
CLASS Total 4.5 (0.6) 3.9 (0.6) 4.7 (0.5) 
(Range) 2.8-6.0 2.8-5.4 3.3-6.0 
Emotional Support 5.7 (0.6) 5.1 (0.6) 5.8 (0.5) 
(Range) 3.6-6.8 3.6-6.1 3.9-6.8 
Positive Climate 5.7 (0.8) 5.0 (0.8) 5.8 (0.7) 
Negative Climate 1.2 (0.3) 1.4 (0.5) 1.2 (0.3) 
Teacher Sensitivity 5.3 (0.9) 4.5 (0.9) 5.5 (0.8) 
Brean JOS tUeent 4.9 (0.9) 4.2 (0.9) 5.0 (0.8) 

Perspectives 

Classroom Organization 5.3 (0.7) 4.5 (0.7) 5.4 (0.6) 
(Range) 2.8-6.9 2.8-6.1 4.0 -6.9 
Behavior Management 5.6 (0.8) 5.0 (0.9) 5.8 (0.6) 
Productivity 5.6 (0.8) 4.9 (0.8) 5.8 (0.6) 
ened Fearning 4.6 (0.8) 3.7 (0.7) 4.8 (0.7) 
Instructional Support 2.7 (0.7) 2.0 (0.6) 2.8 (0.6) 
(Range) 1.2-4.6 1.4-4.0 1.2-4.6 
Concept Development 2.2 (0.7) 1.5 (0.6) 2.3 (0.7) 
Quality of Feedback 2.5 (0.8) 2.0 (0.7) 2.7 (0.8) 
Language Modeling 3.3 (0.9) 2.6 (0.8) 3.4 (0.9) 
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Table 8. ITT Results — Full Sample 


Executive Function 


Language / Literacy 


FDS HTKS WJ PV WJ LW WJ PC 
Model 1 
Intercept 4.16 (0.11) 21.82 (1.69) 93.79 (0.71) 97.55 (0.85) 101.41 (1.11) 
Treatment 0.02 (0.11) 0.35 (1.85) 2.64***(0.77) 3.9***(0.87) -1.46 (1.16) 
County 0.11 (0.09) 0.06 (1.75) 1.78 (0.64) 2.37 (0.69) -0.21 (0.95) 
Fall score 0.56 (0.03) 0.69 (0.04) 0.74 (0.02) 0.85 (0.03) 0.3 (0.04) 
Model 2 
CLASS -0.07 (0.26) -0.37 (4.49) -0.16 (1.83) 1.16 (2.2) 4.48 (2.83) 
English Prof -0.08 (0.21) 2.11 (3.31) 3.61 (1.48) 2.18 (1.67) -1.03 (2.22) 
No PK 0.2 (0.22) 2.17 (3.97) -1.5 (1.59) -3.91 (1.84) -0.64 (2.38) 
CLASS xTrt 0.23 (0.28) 2.74 (4.77) 0.69 (1.93) -0.87 (2.31) -3.09 (2.96) 
Eng Prfx Trt 0.21 (0.22) 0.28 (3.58) -1.89 (1.58) -3.01+ (1.81) 0.65 (2.41) 

Math Behavior Skills 

WJ AP WJ QC SSiS Social Skills SSiS Prob Beh 
Model 1 
Intercept 102.82 (0.94) 94.8 (1.06) 110.18 (1.22) 99.17 (1.52) 
Treatment 0.47 (0.97) 1.06 (1.09) 0.51 (1.25) -2.36 (1.57) 
County 1 (0.79) 2.09 (0.82) 0.62 (1.01) 1.95 (1.27) 
Fall score 0.67 (0.03) 0.75 (0.03) 0.75 (0.04) 0.72 (0.05) 
Model 2 
CLASS 1.88 (2.37) 3.26 (2.74) -0.24 (2.95) -1.44 (3.53) 
English Prof 4.01 (1.84) 3.59 (2.15) -1.55 (2.23) 0.52 (2.65) 
No PK -2.31 (1.93) 0.83 (2.35) 0.17 (2.53) 3.21 (2.89) 
CLASSxTrt = -0.75 (2.48) -3.89 (2.87) 1.14 (3.16) 0.77 (3.73) 
Eng PrfxTrt -4.01 (1.94) -2.91 (2.3) 0.85 (2.5) -1.55 (3.02) 


p-values * p<.05; **p<.01; ***p<.001 
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Figure 1. ITT- Full Sample WJ Picture Vocabulary 


treatment control 


Figure 2. ITT- Full Sample WJ Letter-Word Identification 


treatment control 
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Table 9. TOT Results — Full Sample 


Executive Function 


Language / Literacy 


FDS HTKS WJ PV WJ LW WJ PC 
Model 1 
Intercept 4.24 (0.11) 20.49 (1.71) 93.11 (0.78) 97.35 (0.87) 101.27 (1.14) 
Treatment -0.06 (0.12) 1.44 (1.83) 3.24***(0.83) 4.03***(0.9) -1.36 (1.18) 
County 0.07 (0.1) 0.34 (1.7) 1.99 (0.67) 2.41 (0.72) -0.08 (0.97) 
Fall score 0.57 (0.03) 0.69 (0.04) 0.74 (0.02) 0.86 (0.03) 0.3 (0.04) 
Model 2 
CLASS -0.26 (0.26) -2.22 (4.14) 0.04 (1.8) -0.05 (2.06) 3.28 (2.68) 
English Prof -0.06 (0.23) 4.87 (3.36) 4.71 (1.66) 2.7 (1.72) -0.14 (2.3) 
No PK 0.27 (0.22) 2.2 (3.97) -1.66 (1.68) -4.3 (1.85) -0.71 (2.42) 
CLASS x Trt 0.42 (0.27) 4.74 (4.41) 0.54 (1.9) 0.25 (2.17) -1.85 (2.82) 
Eng Prf x Trt 0.2 (0.24) -2.49 (3.66) -3.01 (1.72) -3.66 (1.86) -0.18 (2.46) 
Math Behavior Skills 

WJ AP WJ QC SSiS Social Skills SSiS Prob Beh 
Model 1 
Intercept 102.56 (0.97) 94.01 (1.08) 110.16 (1.16) 98.72 (1.45) 
Treatment 0.69 (1.01) 1.74 (1.11) 0.46 (1.2) -1.78 (1.5) 
County 1.1 (0.82) 2.17 (0.85) 0.61 (1.01) 2.04 (1.29) 
Fall score 0.66 (0.03) 0.75 (0.03) 0.75 (0.04) 0.71 (0.05) 
Model 2 
CLASS 1.35 (2.28) 2.07 (2.56) 1.03 (2.92) -1.46 (3.4) 
English Prof 4.46 (1.96) 4.31 (2.21) 0.62 (2.34) 0.67 (2.75) 
No PKe -2.41 (2.03) 1.16 (2.32) -0.86 (2.65) 3.87 (2.96) 
CLASS x Trt -0.41 (2.4) -2.64 (2.7) -0.07 (3.05) 0.63 (3.65) 
Eng Prfx Trt  -4.56 (2.04) -3.37 (2.35) -1.57 (2.59) -1.61 (3.12) 


p-values * p<.05; **p<.01; ***p<.001 
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Table 10. ITT Results - DLL Subsample English Outcomes 


Executive Function 


Language / Literacy 


FDS HTKS WJ PV WJ LW WJ PC 
Model 1 
Intercept 4.01 (0.15) 17.64 (3.05) 91.39 (1.66) 90.92 (1.88) 99.25 (1.78) 
Treatment 0.02 (0.14) 4.27 (3.14) 4.11 (1.58) 7.93***(1.83) 0.93 (1.77) 
County -0.1 (0.13) 0.2 (2.97) 0.91 (1.09) 5.09 (1.66) -0.26 (1.23) 
Fall score 0.4 (0.06) 0.63 (0.08) 0.7 (0.03) 0.71 (0.06) 0.06 (0.07) 
Model 2 
CLASS -0.27 (0.36) 5.66 (6.83) -0.8 (3.63) 0.54 (4.03) 2.26 (4.64) 
English Prof -0.3 (0.32) 11.09 (6.03) 6.25 (3.36) -1.71 (3.61) -2.48 (4.16) 
No PK 0.27 (0.34) -8.14 (6.53) -2.6 (3.42) -10.7 (3.85) 1.6 (4.21) 
CLASS x Trt 0.42 (0.37) -2.37 (7.29) 2.13 (3.74) -1 (4.26) -2.18 (4.74) 
Eng Prf x Trt 0.27 (0.34) -4.84 (6.62) -6.51+ (3.52) 2.24 (3.94) 1.86 (4.37) 
Math Behavior Skills 
WJ AP WJ QC SSiS Social Skills SSiS Prob Beh 
Model 1 
Intercept 97.72 (2.07) 90.36 (1.54) 109.27 (2.34) 102.07 (2.83) 
Treatment 5.2 (2.02) 5.27***(1.49) 0.91 (2.27) -3.57 (2.81) 
County 3.22 (1.48) 3.44 (1.31) -0.15(1.84) 2.55 (2.21) 
Fall score 0.62 (0.05) 0.79 (0.05) 0.71 (0.08) 0.69 (0.09) 
Model 2 
CLASS -0.32 (5.36) 0.05 (3.66) 0.36 (5.35) 0.18 (6.93) 
English Prof 0.6 (5.02) 3.03 (3.34) -2.52 (4.99) -2.45 (6.67) 
No PK -5.36 (4.72) 1.95 (3.63) 0.92 (5.64) -2.69 (6.62) 
CLASS x Trt 0.45 (5.47) -1.56 (3.87) 0.88 (5.57) -1.23 (7.1) 
Eng Prf x Trt 1.1 (5.12) -1.05 (3.64) 1.2 (5.26) 1.42 (7.21) 


p-values * p<.05; **p<.01; ***p<.001 
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Figure 3. ITT- DLL Subsample WJ Letter-Word Identification 


treatment control 


Figure 4. ITT- DLL Subsample WJ Quantitative Concepts 


treatment control 
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Table 11. ITT Results - DLL Subsample Spanish Outcomes 


Executive Function 


Language / Literacy 


FDS HTKS BAT PV BAT LW BAT PC 
Model 1 
Intercept 3.42 (0.15) 23.12 (3.65) 70.56 (2.78) 86.16 (1.61) 84.35 (3.18) 
Treatment 0.13 (0.15) 1.57 (3.66) -0.35 (2.93) 2.04 (1.62) 5.32 (3.23) 
County 0.13 (0.13) 2.11 (2.86) 0.3 (2.26) 3.21 (1.38) 3.32 (2.86) 
Fall score 0.49 (0.07) 0.69 (0.07) 0.93 (0.08) 0.6 (0.07) 0.12 (0.09) 
Model 2 
CLASS 0.11 (0.28) 20 (8.01) 3.01 (5.06) 1.5 (3.5) 11.55 (7.41) 
English Prof 0.58 (0.25) -5.58 (7) -7.62 (4.79) 5.64 (3.1) -1.22 (6.73) 
No PK 0.59 (0.29) -5.39 (7.91) -8.79 (5.17) -1.26 (3.59) -12.07 (7.61) 
CLASS x Trt 0.17 (0.3) -13.97 (8.47) -3.55 (5.42) -1.4 (3.72) -11.39 (7.92) 
Eng Prf x Trt -0.26 (0.29) 14.88 (7.6) 11.1 (5.51) -3.19 (3.46) 3.51 (7.56) 
Math 
BAT AP BAT QC 
Model 1 
Intercept 92.11 (1.85) 85.75 (1.59) 
Treatment 2.69 (1.86) 0.32 (1.6) 
County 4.03 (1.67) -0.14 (1.31) 
Fall score 0.67 (0.06) 0.7 (0.06) 
Model 2 
CLASS 11.54 (3.81) 2.91 (3.66) 
English Prof -1.4 (3.34) 1.05 (3.29) 
No PK -4.03 (3.84) 1.68 (3.74) 
CLASS x Trt -10.61 (4.13) -4.05 (3.86) 
Eng Prf x Trt 5.04 (3.67) 0.18 (3.58) 


p-values * p<.05; **p<.01; ***p<.001 
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Table 12. TOT Results - DLL Subsample English Outcomes 


Executive Function 


Language / Literacy 


FDS HTKS WJ PV WJ LW WJ PC 
Model 1 
Intercept 4.07 (0.15) 15.41 (3.01) 89.84 (1.74) 89.79 (1.78) 99.18 (1.74) 
Treatment -0.04 (0.15) 6.35 (3.16) 5.56**(1.68) 8.81***(1.78) 1.1 (1.76) 
County -0.1 (0.13) 0.99 (2.93) 1.24 (1.12) 5.27 (1.68) -0.29 (1.21) 
Fall score 0.4 (0.06) 0.62 (0.08) 0.69 (0.03) 0.69 (0.06) 0.07 (0.07) 
Model 2 
CLASS -0.34 (0.41) 2.99 (8.02) -1.67 (4.22) 0.56 (4.26) 0.87 (5) 
English Prof -0.41 (0.37) 11.03 (6.94) 6.19 (3.86) -1.29 (3.76) -3.93 (4.44) 
No PK 0.37 (0.38) -4.73 (7.63) -2.13 (3.89) -10.87 (4.09) 2.76 (4.44) 
CLASS x Trt 0.52 (0.42) 0.14 (8.37) 2.9 (4.31) -1.33 (4.48) -0.63 (5.09) 
Eng Prf x Trt 0.43 (0.38) -4.66 (7.47) -6.19 (3.99) 1.63 (4.07) 3.58 (4.63) 
Math Behavior Skills 
WJ AP WJ QC SSiS Social Skills | SSiS Prob Beh 
Model 1 
Intercept 97.73 (2.25) 90.13 (1.49) 109.17 (2.18) 101 (2.71) 
Treatment 5.09 (2.23) 5.61***(1.47) 0.97 (2.21) -2.56 (2.82) 
County 3.24 (1.54) 3.16 (1.33) -0.02 (1.83) 2.72 (2.2) 
Fall score 0.62 (0.05) 0.78 (0.06) 0.72 (0.08) 0.69 (0.09) 
Model 2 
CLASS 0.96 (6.5) -0.39 (3.97) 1.66 (5.94) 0.07 (7.57) 
English Prof 0.51 (6.05) 2.66 (3.56) -0.49 (5.43) -0.9 (7.08) 
No PK -7.69 (5.35) 2.15 (3.98) -0.08 (6.1) -2.32 (7.08) 
CLASS x Trt -0.81 (6.59) -1.04 (4.16) -0.22 (6.11) -1.35 (7.71) 
Eng Prf x Trt 0.59 (6.12) -0.21 (3.82) -1.14 (5.66) 0.08 (7.57) 


p-values * p<.05; **p<.01; ***p<.001 
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Table 13. TOT Results - DLL Subsample Spanish Outcomes 


Executive Function 


Language / Literacy 


FDS HTKS BAT PV BAT LW BAT PC 

Model 1 
Intercept 3.52 (0.15) 23.1 (3.6) 70.62 (2.93) 86.31 (1.6) 82.51 (2.96) 
Treatment 0.04 (0.15) 1.55 (3.67) -0.57 (3.08) 1.89 (1.64) 6.92 (3.11) 
County 0.09 (0.14) 2.18 (2.85) 0.37 (2.25) 3.21 (1.36) 4.09 (2.81) 
Fall score 0.48 (0.07) 0.7 (0.07) 0.94 (0.08) 0.6 (0.07) 0.12(0.09) 
Model 2 
CLASS 0.14 (0.31) 17.62 (8.96) 3.5 (5.57) 1.7 (3.76) 9.89 (7.63) 
English Prof 0.53 (0.25) -7.77 (7.21) -7.49 (5.06) 6.78 (3.03) -0.77 (6.47) 
No PK 0.59+ (0.33) -0.22 (8.88) -9.7 (5.83) -2.12 (3.93) -10.99 (8.02) 
CLASS x Trt 0.14 (0.33) -12.09 (9.34) -4.14 (5.88) -1.46 (3.95) -9.83 (8.08) 
Eng Prf x Trt -0.21 (0.29) 17.26 (7.78) 10.43 (5.79) -4.16 (3.39) 3.01 (7.33) 

Math 

BAT AP BAT QC 
Model 1 
Intercept 93.46 (1.84) 85.75 (1.47) 
Treatment 1.42 (1.9) 0.45 (1.53) 
County 3.56 (1.71) -0.09 (1.34) 
Fall score 0.69 (0.06) 0.76 (0.06) 
Model 2 
CLASS 12.17 (4.27) 1.98 (3.56) 
English Prof -1.89 (3.42) -1.38 (2.9) 
No PK -3.83 (4.43) 4.47 (3.8) 
CLASS x Trt -11.52 (4.56) -3.16 (3.78) 
Eng Prf x Trt 5.67 (3.73) 3.15 (3.24) 


p-values * p<.05; **p<.01; ***p<.001 
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Table 14. NC Pre-K Program Characteristics (2017-2018) 


Program Characteristic 


Total NC Pre-K Sites (Centers/Schools) n=1,167 
Total NC Pre-K Classrooms n=2,007 
Total Children Served n=30,035 
Mean (SD) 
Class Size 15.7 (3.5) 
Number of NC Pre-K Children per Class 13.5 (4.2) 
Proportion of NC Pre-K Children per Class 0.86 (0.18) 
Days of Attendance per Child 137 (38.4) 
Days of Operation 170 (13.3) 
NC Star-Rated License % n 
Five-Star 79.9 932 
Four-Star 15.3 178 
Temporary 1.6 19 
Public School in Process 3.3 38 
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Table 15. NC Pre-K Classrooms: Early Childhood Curricula, Formative Assessments, and 
Developmental Screening Measures (2017-2018) 


Educational Resources Used n=2,007 % n 


Early Childhood Curriculum 


Creative Curriculum for Preschool 90.1 1,808 
Opening the World of Learning (OWL) 5.5 111 
High/Scope Preschool Curriculum 2.4 49 
Tools of the Mind 1.2 24 
Investigator Club Prekindergarten Learning System 0.2 3 
Passports: Experiences for Pre-K Success 0.2 3 


Formative Assessment 


Teaching Strategies GOLD / Creative Curriculum Developmental 


Continuum Sate Eee 
Work Sampling System 5.4 109 
COR Advantage / Preschool Child Observation Record (COR) 2.5 50 
Galileo Online Assessment System 0.9 17 
Investigator Club 0.1 2 
Developmental Screening Measure 

Developmental Indicators for the Assessment of Learning (DIAL) 52.9 1,062 
Brigance 38.5 772 
Ages & Stages Questionnaire (ASQ) 6.9 140 


Parents’ Evaluation of Developmental Status (PEDS) 1.6 33 
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Table 16. Distribution of NC Pre-K Classrooms by Setting Type (2017-2018) 


Setting Type n=2,007 
Public School 
Private 
Private For-Profit 
Private Non-Profit 
Head Start 
Head Start Not Administered by Public School 


Head Start Administered by Public School 
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% 


52.1 


33.0 


25.3 


7.7 


14.9 


10.7 


4.2 


Table 17. Characteristics of NC Pre-K Children (2017-2018) 


Characteristic n=30,035 %/Mean n 
Child’s age on 8/31 of program year 45 30,035 
Gender 
Male 50.6% 15,210 
Female 49.4% 14,825 
Race 
White/European-American 48.0% 14,409 
Black/African-American 36.2% 10,866 
Native American/Alaskan Native 4.5% 1,350 
Multiracial 7.0% 2,113 
Asian 2.9% 885 
Native Hawaiian/Pacific Islander 1.4% 412 
Ethnicity 
Non-Hispanic/Latino 76.1% 22,866 
Hispanic/Latino 23.9% 7,169 


Parents Employed 


Mother 51.4% 15,435 
Father 48.6% 14,599 
Mother and/or Father 78.8% 23,658 
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Table 18. Eligibility Factors for NC Pre-K Children (2017-2018) 


Eligibility Factors? n=30,035 % n 
Family Income 


130% of poverty and below 70.4 21,134 
(eligible for free lunch) 


131-185% of poverty 17.1 5,125 
(eligible for reduced-price lunch) 


186-200% of poverty 3.3 991 
201-250% of poverty 4.0 1,201 
>251% of poverty 5.3 1,584 


Limited English Proficiency 


Family and/or child speak limited or 18.8 5,660 
no English in the home 


Educational Need 


Indicated by performance on approved developmental 22.0 6,602 
screening or an IEP 


Identified Developmental Disability 
Child has been screened and evaluated 5.3 1,579 


Chronic Health Condition(s) 


Indicated by diagnosis from a professional health care provider 4.0 1,186 
Military Family 
Parent on active duty or injured, receiving military disability 5.1 1,538 


retirement, or killed on active duty 


2 Children are eligible for the NC Pre-K Program primarily based on age and family income. Children must be four years old by August 31 of the program 
year, with a gross family income up to 75% of state median income. Children who do not meet the income eligibility may be eligible if they have at least 
one of the following: limited English proficiency, identified disability, chronic health condition, educational need, or a parent serving in the military. 
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Table 19. Prior Placement for NC Pre-K Children (2017-2018) 


Prior Placement n=30,035 % n 


Children who have never been served in any preschool or child care 
58.0 17,422 


setting 
Children who are currently unserved (may previously have been in 13.8 4,141 
preschool or child care) 
Children who are in unregulated child care 2.4 727 
Children who are not receiving subsidy but in a regulated preschool or 

‘ 15.3 4,597 
child care program 
Children who are receiving subsidy and in regulated child care or 99 2.969 
preschool program ; is 
Not reported 0.6 179 
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Table 20. Education Levels of NC Pre-K Lead Teachers (2017-2018) 


Highest Education Level 


MA/MS or higher BA/BS AA/AAS HS Diploma/GED 
Setting Type Total n4 % n % n % n % n 
Public School 1,147 17.7 203 82.2 943 0 0 0.1 1 
Private 916 8.6 79 90.1 825 0.1 1 0 0 
All 2,063 13.7 283 85.7 1,768 0.1 1 0.1 1 


Setting Type? 
Public School 
Private 


All 


Table 21. Licensure/Credential Levels of NC Pre-K Lead Teachers (2017-2018) 


Highest Licensure/Credential 


Other Teacher’s 


B-K License® License® CDA Credential’ NCECC8 None 
Total n % n % n % n % n % n 
1147 96.3 1,104 3.3 38 0 0 0 0 0.4 5 
916 85.5 783 2.6 24 0.2 2 0.8 7 10.9 100 
2,063 91.5 1,887 3.0 62 0.1 2 0.3 7 5.1 105 


3 Teachers in Head Start classrooms administered by public schools are included in public school setting types; teachers in Head 
Start classrooms not administered by public schools are included in private setting types. 
4 Data were not reported for 24 teachers. 
5 B-K = Birth-Kindergarten license. This category includes teachers with a B-K license, Initial B-K license (formerly SP I), Continuing 
B-K license (formerly SP Il), Lateral Entry B-K license, Provisional B-K license, or Preschool Add-on license. 

6 Other teacher’s license includes non-early childhood licenses and licenses from other states. 
7 CDA = Child Development Associate. 
8 NCECC = North Carolina Early Childhood Credential. 
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Table 22. Pre-K Classrooms by Setting Type (2003-04 — 2017-18) 


2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014 2014-2015 2015-2016 2016-2017 2017-2018 


Setting Type n=866 n=1,027 n=1,218 n=1,439 n=2,110 n=2,322 n=2,308 n=2,262 n=2,057 n=2,150 n=1,993 n=1,974 n=1,962 n=1,949 n=1,949 
Public Preschool 49.7% 54.1% 53.0% 55.0% 53.4% 51.9% 52.2% 54.1% 50.6% 50.7% 54.2% 51.6% 51.6% 52.3% 52.1% 
(430) (556) (646) (791) (1,127) (1,205) (1,205) (1,223) (1,041) (1,090) (1,080) (1,019) (1,013) (1019) (1046) 
Private 35.2% 34.8% 35.1% 32.0% 28.5% 28.8% 28.1% 27.1% 33.3% 33.5% 31.9% 33.2% 32.9% 32.5% 33.0% 
(305) (357) (427) (461) (602) (669) (649) (613) (686) (719) (636) (655) (645) (633) (662) 
Private For-Profit 25.1% 24.1% 23.6% 21.3% 19.4% 20.1% 19.3% 18.7% 24.2% 24.3% 23.4% 24.9% 25.2% 24.9% 25.3% 
(217) (247) (287) (306) (409) (467) (446) (424) (497) (522) (466) (491) (494) (485) (507) 
Private Non-Profit 10.2% 10.7% 11.5% 10.8% 9.1% 8.7% 8.8% 8.4% 9.2% 9.2% 8.5% 8.3% 7.7% 7.6% 7.7% 
(88) (110) (140) (155) (193) (202) (203) (189) (189) (197) (170) (164) (151) (148) (155) 
Head Start 15.1% 11.1% 11.9% 13.0% 18.1% 19.3% 19.7% 18.8% 16.0% 15.8% 13.9% 15.2% 15.5% 15.2% 14.9% 
(131) (114) (145) (187) (381) (448) (454) (426) (330) (341) (277) (300) (304) (297) (299) 
Head Start Not 9.2% 8.4% 9.0% 10.1% 14.8% 15.8% 15.8% 14.9% 12.4% 12.8% 10.6% 10.6% 10.7% 10.4% 10.7% 
Administered by (80) (86) (110) (145) (313) (366) (364) (338) (256) (276) (212) (209) (209) (202) (214) 
Public School 
Head Start 5.9% 2.7% 2.9% 2.9% 3.2% 3.5% 3.9% 3.9% 3.6% 3.0% 3.3% 4.6% 4.8% 4.9% 4.2% 
Administered by (51) (28) (35) (42) (68) (82) (90) (88) (74) (65) (65) (91) (95) (95) (85) 
Public School 
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Table 23. Prior Placement of Pre-K Children (2003-04 — 2017-18) 


2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014 2014-2015 2015-2016 2016-2017 2017-2018 
Prior Placement n=10,891 n=13,515  n=17,251 n=20,468 n=29,978 n=33,798 n=34,212 n=33,747 = n=29,311 =n=32,142 =n=29,346 n=29,271 n=28,757 n=28,905 n=30035 


Children who have never 62.3% 60.4% 59.9% 58.8% 54.6% 54.0% 54.8% 57.5% 59.6% 59.5% 61.7% 57.7% 59.4% 59.2% 58.0% 


been served in any preschool (6,788) (8,165) (10,325) (12,033) (16,353) (18,237) (18,755) (19,397) (17,484) (19,120) (18,111) (16,904) (17,069) (17,114) (17,422) 
or child care setting 


Children who are currently 20.9% 17.9% 13.2% 13.1% 13.1% 16.1% 15.1% 14.6% 17.9% 19.2% 16.1% 13.9% 14.4% 14.3% 13.8% 
unserved (may previously (2,282) (2,418) (2,270) (2,676) (3,938) (5,433) (5,155) (4,918) (5,234) (6,181) (4,729) (4,055) (4,131) (4,125) (4,141) 
have been in preschool or 

child care) 

Children who are in -- 4.5% 4.2% 4.0% 5.3% 5.9% 4.7% 3.8% 2.8% 2.0% 1.8% 2.2% 1.8% 1.9% 2.4% 

unregulated child care (608) (716) (814) (1,592) (1,981) (1,609) (1,291) (810) (647) (520) (646) (509) (546) (727) 

Children who are not 5.6% 3.4% 2.1% 2.4% 3.6% 4.5% 4.7% 5.2% 13.5% 12.0% 13.4% 17.2% 15.5% 14.5% 15.3% 
receiving subsidy, but are ina (606) (463) (364) (497) (1,072) (1,510) (1,612) (1,765) (3,955) (3,845) (3,928) (5,022) (4,460) (4,193) (4,597) 


regulated preschool or child 
care program 


Children who are receiving -- -- -- -- -- -- -- -- 6.2% 7.3% 7.0% 8.8% 8.6% 9.7% 9.9% 
subsidy and in regulated child (1,828) (2,349) (2,058) (2,575) (2,474) (2,794) (2,969) 
care or preschool program 

Children served for 5 months -- 3.2% 5.9% 4.1% 3.9% 2.3% 2.1% 1.5% -- -- -- -- -- -- 

or less in the year prior to (436) (1,022) (849) (1,161) (780) (721) (520) 


More at Four program in any 
preschool or child care 


Other children, including 11.2% 10.5% 7.2% 7.2% 8.5% 4.6% 4.4% 4.5% - -- -- -- - 
those in pre-kindergartens or (1,215) (1,425) (1,236) (1,474) (2,556) (1,570) (1,507) (1,527) 

child care settings that do not 

meet More at Four program 


standards. 
Children served by this site as -- -- 7.6% 10.4% 11.0% 12.7% 14.2% 12.8% -- -- -- -- -- -- 
3-year-olds. (1,318) (2,125) (3,306) (4,287) (4,853) (4,329) 
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Table 24. Education Levels of Pre-K Lead Teachers (2003-04 — 2017-18) 


Highest Education Level 


MA/MS or higher BA/BS AA/AAS HS diploma/GED 

Setting Type? Total n> % n % n % n % n 
2003-2004 

Public School 450 17.1 77 77.1 347 2.4 11 3.3 15 

Private 534 4.1 22. 62.5 334 25.3 135 8.1 43 

All 984 10.1 99 69.2 681 14.8 146 5.9 58 
2004-2005 

Public School 615 15.1 93 83.6 514 1.0 6 0.3 2 

Private 519 4.2 22 61.3 318 29.5 153 5.0 26 

All 1,133 10.2 115 73.3 831 14.0 159 2.5 28 
2005-2006 

Public School 725 13.8 100 84.6 613 1.4 10 0.3 2 

Private 620 3.4 21 61.0 378 31.8 197 3.9 24 

All 1,342 9.0 121 73.7 989 15.4 206 1.9 26 
2006-2007 

Public School 875 15.1 132 84.0 735 0.8 7 0.1 1 

Private 684 4.4 30 57.9 396 34.2 234 3.5 24 

All 1,555 10.4 162 72.5 1,128 15.4 240 1.6 25 
2007-2008 

Public School 1,197 13.8 165 84.5 1,012 1.5 18 0.2 2 

Private 990 3.8 38 50.0 495 41.8 414 4.3 43 

All 2,183 9.3 203 68.9 1,503 19.8 432 2.1 45 
2008-2009 

Public School 1,305 14.9 195 83.5 1,090 1.4 18 0.2 2 

Private 1,109 4.2 47 52.4 581 41.3 458 21 23 

All 2,409 10.0 241 69.2 1,667 19.8 476 1.0 25 
2009-2010 

Public School 1,308 15.3 200 83.0 1,085 1.8 23 0.0 0 

Private 1,107 5.3 59 62.2 689 31.7 351 0.7 8 

All 2,412 10.7 259 73.5 1,772 15.5 373 0.3 
2010-2011 

Public School 1,333 16.0 213 82.9 1,105 1.1 15 0.0 0 

Private 1,065 7.2 77 73.9 787 18.8 200 0.1 1 

All 2,395 12.1 289 78.9 1,889 9.0 216 0.0 1 
2011-2012 

Public School 1,142 15.8 181 83.7 956 0.4 5 0.0 0 

Private 1,054 8.6 91 87.3 920 3.6 38 0.5 5 

All 2,191 12.4 271 85.4 1,872 2.0 43 0.2 5 
2012-2013 

Public School 1,191 16.3 194 83.5 995 0.2 2 0.0 0 

Private 1,064 7.9 84 89.9 957 2.1 22 0.1 1 

All 2,255 12.3 278 86.6 1,952 1.1 24 0.0 1 


2 Teachers in Head Start classrooms administered by public schools are included in public school setting types; teachers in Head Start classrooms not 
administered by public schools are included in private setting types. 

>In some cases, the n for All is less than the sum of the n’s for Public School and Private because some teachers worked in both public and private settings 
(n=1 in 2004-2005; n=3 in 2005-2006 and 2009-2010; n=4 in 2006-2007, 2007-2008, and 2010-2011; and n=5 in 2008-2009 and 2011-2012). 
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Table 24 (Cont) Education Levels of Pre-K Lead Teachers (2003-04 — 2017-18) 


Highest Education Level 


MA/MS or higher BA/BS AA/AAS HS diploma/GED 

Setting Type? Total n> % n % n % n % n 
2013-2014 

Public School 1,168 15.4 180 84.4 985 0.2 2 0.0 0 

Private 932 11.2 104 88.0 819 1.0 9 0.0 0 

All 2,099 13.6 285 85.9 1,803 0.5 11 0.0 0 
2014-2015 

Public School 1,149 19.4 223 80.4 924 0.1 1 0.1 1 

Private 911 10.0 92 90.0 819 0.0 0 0.0 0 

All 2,060 15.3 315 84.7 1,743 0.0 1 0.0 ek 
2015-2016 

Public School 1,125 18.3 206 81.4 916 0.0 0 0.3 3 

Private 881 10.1 89 89.7 790 0.1 al 0.1 

All 2,006 14.7 295 85.0 1,706 0.0 1 0.2 4 
2016-2017 

Public School 1135 17.6 200 82.1 932 0.1 1 0.2 2 

Private 864 9.4 81 90.2 779 0.3 3 0.1 1 

All 1999 14.1 281 85.6 1,711 0.2 4 0.1 3 
2017-2018 

Public School 1,147 17.7 203 82.2 943 0 0 0.1 1 

Private 916 8.6 79 90.1 825 0.1 1 0 0 

All 2,063 13.7 283 85.7 1,768 0.1 1 0.1 1 
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Table 25. Licensure/Credential Levels of Pre-K Lead Teachers (2003-04 — 2017-18) 


Highest Licensure/Credential 


B-K License? Other Teacher’s License? CDA Credential‘ NCECC¢ None 

Setting Type® Total nf % n % n % n % n % n 
2003-2004 

Public School 454 68.1 309 18.3 83 0.0 0 1.1 5 12.6 57 

Private 535 16.4 88 10.5 56 34 21 16.3 87 ae. 283 

All 989 40.1 397 14.1 139 2.1 21 9.3 92 34.4 340 
2004-2005 

Public School 615 75.4 464 13.5 83 0.7 4 1.1 7 9.3 57 

Private 519 15.2 7 91 47 9.6 50 28.9 150 37.2 193 

All 1,133 47.8 542 11.5 130 4.8 54 13.9 157 22.1 250 
2005-2006 

Public School 725 83.1 601 9.8 71 0.6 4 1.1 8 5.7 41 

Private 620 16.5 103 8.5 53 6.5 40 31.5 195 36.9 229 

All 1,342 52.3 702 9.2 124 3.3 44 15.1 202 20.0 269 
2006-2007 

Public School 875 86.2 753 8.0 70 0.6 5 4,3 11 4.1 36 

Private 684 20.6 142 te0 51 5.6 38 32.3 221 33.9 232 

All 1,555 57.4 893 7.7 120 2.8 43 14.9 231 17.2 268 
2007-2008 

Public School 1,197 85.7 1,025 7.2 86 0.9 11 1.1 13 5.2 62 

Private 990 17.1 172 5.7 56 6.5 64 379: 375 32.6 323 

All 2,183 54.7 1,194 6.5 142 3.4 75 17.7 387 17.6 385 
2008-2009 

Public School 1,305 86.8 1,134 7.5 98 0.6 8 LZ 16 3.8 49 

Private 1,109 22.7 256 5.8 64 4.4 49 39.2 435 27.5 305 

All 2,409 57.5 1,385 6.7 162 2.4 oe 18.7 451 14.7 354 
2009-2010 

Public School 1,308 88.5 1,156 7.0 91 0.5 6 1.9 25 2.3 30 

Private 1,107 30.8 341 7.6 84 4.6 al 32.9 364 24.1 267 

All 2,412 62.0 1,496 7.3 175 2.3 56 16.1 388 12.3 297 


*B-K = Birth-Kindergarten license. This category includes teachers with a B-K license, Initial B-K license (formerly SP I), Continuing B-K license (formerly SP II), Lateral Entry B-K license, Provisional B-K license, or Preschool Add-on license. 

> Other teacher’s license includes non-early childhood licenses and licenses from other states. 

¢ CDA = Child Development Associate. 

4 NCECC = North Carolina Early Childhood Credential. 

© Teachers in Head Start classrooms administered by public schools are included in public school setting types; teachers in Head Start classrooms not administered by public schools are included in private setting types. 

In some cases, the n for All is less than the sum of the n’s for Public School and Private because teachers worked in both setting types (n=1 in 2004-2005; n=3 in 2005-2006 and 2009-2010; n=4 in 2006-2007, 2007-2008, and 2010-2011; and n=5 in 
2008-2009 and 2011-2012). 
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Table 25 (Continued) Licensure/Credential Levels of Pre-K Lead Teachers (2003-04 — 2017-18) 


Highest Licensure/Credential 


B-K License> Other Teacher’s License CDA Credential NCECC None 

Setting Type® Total n¢ % n % n % n % n % 
2010-2011 

Public School 1,333 92.8 1,237 4.6 61 0.2 3 1.3 17 1.1 15 

Private 1,065 44.0 471 9.2 98 2.9 31 22.6 241 21.0 224 

All 2,394 71.2 1,704 6.6 159 1.4 34 10.8 259 10.0 239 
2011-2012 

Public School 1,142 91.3 1,043 6.0 68 0.1 1 0.7 8 1.9 22 

Private 1,054 51.0 538 11.0 116 1.4 15 12.9 135 23.7 250 

All 2,191 72.0 1,578 8.4 183 0.7 16 6.5 143 12.4 271 
2012-2013 

Public School 1,191 92.9 1,106 4.9 58 0.1 1 0.3 3 1.9 23 

Private 1,064 57.0 606 9.0 96 0.9 10 11.2 119 21.9 233 

All 2,255 75.9 1,712 6.8 154 0.5 11 5.4 122 11.4 256 
2013-2014 

Public School 1,168 93.7 1,093 5.1 59 0.1 0.1 1 1.2 14 

Private 932 63.8 594 10.5 98 0.9 8 6.3 59 18.6 173 

All 2,099 80.3 1,686 7.5 157 0.4 9 2.9 60 8.9 187 
2014-2015 

Public School 1,149 91.7 1,054 L.7: 20 0.0 0 0.5 6 6.0 69 

Private 911 74.5 679 6.3 57 0.1 4.3 39 14.8 135 

All 2,060 84.1 1,733 3.8 77 0.0 1 2.2 45 9.9 204 
2015-2016 

Public School 1,125 96.0 1,080 2.7 30 0.0 0 0.0 0 1.3 15 

Private 881 76.7 676 4.2 37 0.6 5 2.2 19 16.3 144 

All 2,006 87.5 1,756 3.3 67 0.2 5 0.9 19 7.9 159 
2016-2017 

Public School 1,136 96.2 1093 3.3 38 0.0 0 0.1 1 0.4 4 

Private 887 80.2 711 5.1 45 0.3 1.1 10 13.3 118 

All 2023 89.2 1804 4.1 83 0.1 3 0.5 11 6.0 122 
2017-2018 

Public School 1147 96.3 1,104 3.3 38 0 0 0 0 0.4 5 

Private 916 85.5 783 2.6 24 0.2 0.8 10.9 100 

All 2,063 91.5 1,887 3.0 62 0.1 2 0.3 5.1 105 
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Figure 5. NC Pre-K Setting Types by Cohort 
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Figure 6. Prior Placement for NC Pre-K Children by Cohort 
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Figure 7. NC Pre-K Teacher Qualifications by Cohort 
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Appendix 1 
List of NC Pre-K Evaluation Reports 


Peisner-Feinberg, E. S. (2003). Child and program characteristics of the North Carolina More at Four Pre- 
kindergarten Program: Year 1 (January-June 2002). Chapel Hill: The University of North Carolina, FPG 
Child Development Institute. 


Peisner-Feinberg, E. S., & Maris, C. L. (2005). Evaluation of the North Carolina More at Four Pre-kindergarten 
Program: Year 2 (July 1, 2002—June 30, 2003). Chapel Hill: The University of North Carolina, FPG Child 
Development Institute. 


Peisner-Feinberg, E. S., & Maris, C. L. (2005). Evaluation of the North Carolina More at Four Pre-kindergarten 
Program: Year 3 Report (July 1, 2003—June 30, 2004). Chapel Hill: The University of North Carolina, 
FPG Child Development Institute. 


Peisner-Feinberg, E. S., & Maris, C. L. (2006). Evaluation of the North Carolina More at Four Pre-kindergarten 
Program: Children’s longitudinal outcomes and classroom quality in kindergarten. Chapel Hill: The 
University of North Carolina, FPG Child Development Institute. 


Peisner-Feinberg, E. S., Elander, K.C., & Maris, C. L. (2006). Evaluation of the North Carolina More at Four Pre- 
kindergarten Program: Year 4 (July 1, 2004—June 30, 2005) Program characteristics and services. 
Chapel Hill: The University of North Carolina, FPG Child Development Institute. 


Peisner-Feinberg, E. S., & Schaaf, J. M. (2007). Evaluation of the North Carolina More at Four Pre-kindergarten 
Program: Children’s outcomes and program quality in the fifth year. Chapel Hill: The University of 
North Carolina, FPG Child Development Institute. 


Peisner-Feinberg, E. S., & Schaaf, J. M. (2008). Evaluation of the North Carolina More at Four Pre-kindergarten 
Program: Children’s longitudinal outcomes and program quality over time (2003-2007). Chapel Hill: 
The University of North Carolina, FPG Child Development Institute. 


Peisner-Feinberg, E. S., & Schaaf, J.M. (2008). Evaluation of the North Carolina More at Four Pre-kindergarten 
Program: Performance and progress in the seventh year (2007-2008). Chapel Hill: The University of 
North Carolina, FPG Child Development Institute. 


Peisner-Feinberg, E. S. & Schaaf, J. M. (2009). Evaluation of the North Carolina More at Four Pre-kindergarten 
Program: A look across time at children’s outcomes and classroom quality from pre-k through 
kindergarten (2003-2009). Chapel Hill: The University of North Carolina, FPG Child Development 
Institute. 


Peisner-Feinberg, E. S., & Schaaf, J.M. (2010). Long-term effects of the North Carolina More at Four Pre- 
kindergarten Program: Children’s reading and math skills at third grade. Chapel Hill: The University of 
North Carolina, FPG Child Development Institute. 


Peisner-Feinberg, E. S., & Schaaf, J.M. (2011). Effects of the North Carolina More at Four Pre-kindergarten 
Program on children’s school readiness skills: Key findings. Chapel Hill: The University of North 
Carolina, FPG Child Development Institute. 


Peisner-Feinberg, E. S., Schaaf, J. M., Hildebrandt, L., & LaForett, D. R. (2013). Quality and characteristics of 
the North Carolina Pre-Kindergarten Program: 2011-2012 statewide evaluation. Chapel Hill: The 
University of North Carolina, FPG Child Development Institute. 
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Peisner-Feinberg, E. S., LaForett, D. R., Schaaf, J. M., Hildebrandt, L. M., Sideris, J., & Pan, Y. (2014). Children’s 
outcomes and program quality in the North Carolina Pre-Kindergarten Program: 2012-2013 Statewide 
evaluation. Chapel Hill: The University of North Carolina, FPG Child Development Institute. 


Peisner-Feinberg, E. S., Schaaf, J. M., Hildebrandt, L. M., Pan, Y. & Warnaar, B. L. (2015). Children’s 
kindergarten outcomes and program quality in the North Carolina Pre-Kindergarten Program: 2013- 
2014 statewide evaluation. Chapel Hill: The University of North Carolina, FPG Child Development 


Institute. 
Peisner-Feinberg, E. S., Garwood, J. D., & Mokrova, I. L. (2016). Children’s pre-k experiences and outcomes in 


the North Carolina Pre-Kindergarten Program: 2014-2015 statewide evaluation. Chapel Hill, NC: The 
University of North Carolina, FPG Child Development Institute. 


Peisner-Feinberg, E. S., Mokrova, I. L., & Anderson, T. L. (2017). Effects of participation in the North Carolina 
Pre-Kindergarten Program at the end of kindergarten: 2015-2016 statewide evaluation. Chapel Hill, 
NC: The University of North Carolina, FPG Child Development Institute. 


Peisner-Feinberg, E. S., Van Manen, K.W., & Mokrova, |. L. (2018). Variations in enrollment practices in the NC 
Pre-K Program: 2016-2017 statewide evaluation. Chapel Hill, NC: The University of North Carolina, 
FPG Child Development Institute. 


Reports can be downloaded at: https://fpg.unc.edu/resource-list/750 


54 


Appendix 2 


Technical Information for Analysis Approach 


Constructions of Weights to Account for Potential Bias 
Baseline Non-equivalence & Inverse Probability of Treatment Weights (IPTW) 


First, we evaluated baseline equivalence among children randomized to the treatment and control 
groups by comparing whether distributions of demographic characteristics were exchangeable (i.e., 
balanced) between groups. Randomization occurred differently in the two counties, which resulted in 
potential confounding bias due to lack of baseline equivalence (i.e., exchangeability) between treatment 
and comparison groups. Adjusting for confounding using inverse probability of treatment weights 
(IPTW) creates a pseudo-population in which treatment assignment is independent of measured 
confounding covariates, allowing estimation of a consistent causal effect of NC Pre-K exposure on child 
assessment outcomes. After an evaluation of baseline equivalence among selected children who were 
eligible and randomized, the following characteristics were imbalanced across treatment and control 
groups after randomization: county, race, limited English proficiency, educational need, and 
Hispanic/Latino ethnicity. (See Table 26.) Given the variation in demographic distribution between the 
two participating counties, and the way in which randomization was conducted, these differences were 
expected. 


We constructed stabilized IPTW to create balance between the treatment and control groups at the 
point of randomization. For individuals who received treatment, the stabilized IPTW were comprised of 
a numerator representing the probability of treatment in the population, and a denominator which was 
the probability that each individual would receive the treatment they actually received, modeled using 
logistic regression, conditional on a set of measured confounding variables (i.e., county, race, limited 
English, educational need and Hispanic/Latino ethnicity). For children randomized to NC Pre-K, weights 
reflected the probability of being randomly assigned to treatment divided by the probability of being 
assigned to treatment given the factors listed above. For children in the control group, their IPTW 
reflected the probability of being randomly assigned to control divided by the probability of being 
assigned to control given the factors listed above. 


i ne Pr[NCPreK =1] 
ne if (NCPreK=1): Pr[NCPreK=1|confounding variables]’ oe 


1- Pr[NCPreK=1] 


if (NCPreK=0): 1— Pr[NCPreK=1|confounding variables] 


Selection Bias & Inverse Probability of Censoring Weights (IPCW) 


Second, we evaluated possible selection bias by comparing distributions of measured characteristics of 
children and families who were recruited and enrolled in the evaluation study compared to all children 
who were randomized (i.e., comparing participants with the entire randomized population regardless of 
participation). The timing of randomization occurred prior to recruitment into the study, resulting in a 
randomized population that would largely not participate in the evaluation assessments (i.e., who were 
censored/lost after randomization). Children participating in the study differed from the entire group of 
children who were randomized in their race, family size and treatment assignment at randomization. 
(See Table 27.) 
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To account for the expected differences between participants and all randomized children (i.e., selection 
bias), we constructed inverse probability of censoring/selection weights (IPCW). The IPCW do not 
eliminate the effect of censoring. Instead, these weights assume that censoring (i.e., non-participation) 
occurs at random, conditional on measured covariates. For individuals who participated in the study, 
the IPCW were comprised of a numerator that was the probability of participating among all 
randomized children. The denominator reflected the probability that each individual would participate, 
modeled using logistic regression, conditional on a set of measured confounding variables (race, family 
size, and treatment assignment). Participating children were assigned a stabilized IPCW such that the 
marginal probability of participation among all randomized children was divided by the probability that 
each individual participated conditional on the factors associated with participation. Non-participating 
children received a weight of zero, since they provided no outcome data to the study 


Pr[Participant=1] 4 
Pr[Participant=1|confounding variables]’ 


IPCW if (Participant=1): W= r 


if (NCPreK=0), W=0 


The IPTW and IPSW were multiplied and added to the statistical analyses of treatment effects on 
children’s outcomes. 


Statistical Comparison Models 


Proc Mixed in SAS 9.4 (SAS Institute, Cary, NC, USA) was used for model specification and analysis, and 
Proc Impute and Proc MlAnalyze for conducting and analyzing the results of the multiple imputations. 
Two models were used to evaluate the impact of NC Pre-K on children’s outcomes at the end of the pre- 
k year. 


The first model examined the most direct impact of treatment with only treatment, county and previous 
fall scores included as predictors. 


Model 1 
Level 1 (for child i in classroom j) 

Yi = Bo + Bay Treatment) + Ba Fall score + ej 
Level 2 (for classroom j) 

Boj = Po + P1t+ Countyj++ Zo; 


Bij = Pit 23; 
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The second model asked whether classroom quality or English (or Spanish) language proficiency 
moderated treatment effects. 


Model 2 


Level 1 (for child i in classroom j) 


Yi = Bo, + Bay Treatment; + Ba Fall score; + Ba; English Proficiency; + Ba English Proficiency; x Treatment; 
ej 


Level 2 (for classroom j) 
Boj = Poo + P10 County; ++ P20 CLASS; ++ p3o AnyPKj ++ Zo; 
Baj = Pio + P1iCLASS total + 023; 
B2j = P20 


B3j = P30 
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Table 26. Baseline Equivalence Results for Full Randomization Sample 


Variables 
Family size 
Mean (SD) 
Min-Max 
Female 
0 
1 
Hispanic/Latino 
0 
1 
Black/African American 


White 


Military parent 

0 

1 
<75% SMI 

0 

1 
Educational need 


IEP 


Limited English proficiency 


0 
1 
Poverty status 

Below 130% 
131-185% 
186-200% 
201-250% 
251-300% 
Over 300% 


All 
(N=2243) 


3.9(1.3) 
1.0 - 13.0 


1012 (45.2) 
1227 (54.8) 


1505 (67.3) 
732 (32.7) 


973 (43.4) 
1270 (56.6) 


1380 (61.5) 
863 (38.5) 


2172 (98.2) 
39 (1.8) 


286 (12.8) 
1957 (87.2) 


1696 (76.2) 
529 (23.8) 


1440 (96.8) 
47 (3.2) 


1253 (56.0) 
985 (44.0) 


1367 (61.0) 
448 (20.0) 
89 (4.0) 
113 (5.0) 
30 (1.3) 
192 (8.6) 


Control 
(N=520) 


3.9(1.2) 
LOe76 


231 (44.7) 
286 (55.3) 


366 (70.4) 
154 (29.6) 


240 (46.2) 
280 (53.8) 


351 (67.5) 
169 (32.5) 


495 (97.2) 
14 (2.8) 


71 (13.7) 
449 (86.3) 


451 (87.9) 
62 (12.1) 


230 (93.5) 
16 (6.5) 


313 (60.4) 
205 (39.6) 


319 (61.3) 
98 (18.8) 
20 (3.8) 
27 (5.2) 

8 (1.5) 

48 (9.2) 
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Treatment 
(N=1723) 


3.9(1.3) 
1.0 - 13.0 


781 (45.4) 
941 (54.6) 


1139 (66.3) 
578 (33.7) 


733 (42.5) 
990 (57.5) 


1029 (59.7) 
694 (40.3) 


1677 (98.5) 
25 (1.5) 


215 (12.5) 
1508 (87.5) 


1245 (72.7) 
467 (27.3) 


1210 (97.5) 
31 (2.5) 


940 (54.7) 
780 (45.3) 


1048 (60.9) 
350 (20.3) 
69 (4.0) 

86 (5.0) 

22 (1.3) 
144 (8.4) 


p 
Value 
0.3479 


0.7873 


0.0848 


0.1452 


0.0014 


0.0540 


0.4812 


<.0001 


0.0010 


0.0203 


0.9476 


Absolute 
Standardized 
Difference 


0.0 


0.0 
0.0 


0.1 
0.1 


0.1 
0.1 


0.2 
0.2 


0.1 
0.1 


0.0 
0.0 


0.4 
0.4 


0.2 
0.2 


0.1 
0.1 


0.0 
0.0 
0.0 
0.0 
0.0 
0.0 


Table 27. Baseline Equivalence Results for Study Participants vs Non-Participants 


Absolute 
All Non-Participants Participants P Standardized 
Variables (N=2243) (N=1661) (N=582) Value Difference 
Family size 0.1057 
Mean (SD) 3.9(1.3) 4.0(1.3) 3.8(1.3) 0.2 
Min-Max 1.0 - 13.0 1.0 - 13.0 1.0 - 8.0 
Female 0.6556 
0 1012 (45.2) 754 (45.5) 258 (44.4) 0.0 
1 1227 (54.8) 904 (54.5) 323 (55.6) 0.0 
Hispanic/Latino 0.5007 
0 1505 (67.3) 1120 (67.7) 385 (66.2) 0.0 
1 732 (32.7) 535 (32.3) 197 (33.8) 0.0 
Black/African American 0.9637 
0 973 (43.4) 721 (43.4) 252 (43.3) 0.0 
1 1270 (56.6) 940 (56.6) 330 (56.7) 0.0 
White 0.2006 
0 1380 (61.5) 1009 (60.7) 371 (63.7) 0.1 
1 863 (38.5) 652 (39.3) 211 (36.3) 0.1 
Military parent 0.4196 
0 2172 (98.2) 1602 (98.1) 570 (98.6) 0.0 
1 39 (1.8) 31 (1.9) 8 (1.4) 0.0 
<75% SMI 0.1055 
0 286 (12.8) 223 (13.4) 63 (10.8) 0.1 
1 1957 (87.2) 1438 (86.6) 519 (89.2) 0.1 
Educational need 0.1128 
0 1696 (76.2) 1243 (75.4) 453 (78.6) 0.1 
1 529 (23.8) 406 (24.6) 123 (21.4) 0.1 
IEP 0.0748 
0 1440 (96.8) 1084 (97.3) 356 (95.4) 0.1 
1 47 (3.2) 30 (2.7) 17 (4.6) 0.1 
Limited English 0.2169 
proficiency 
0 1253 (56.0) 915 (55.2) 338 (58.2) 0.1 
1 985 (44.0) 742 (44.8) 243 (41.8) 0.1 
Randomized to NC Pre-K 0.0031 
0 520 (23.2) 411 (24.7) 109 (18.7) 0.1 
1 1723 (76.8) 1250 (75.3) 473 (81.3) 0.1 
Poverty status 0.3951 
Below 130% 1367 (61.0) 1005 (60.6) 362 (62.2) 0.0 
131-185% 448 (20.0) 335 (20.2) 113 (19.4) 0.0 
186-200% 89 (4.0) 68 (4.1) 21 (3.6) 0.0 
201-250% 113 (5.0) 76 (4.6) 37 (6.4) 0.1 
251-300% 30 (1.3) 25 (1.5) 5 (0.9) 0.1 
Over 300% 192 (8.6) 148 (8.9) 44 (7.6) 0.0 
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